Malware Analyst’s Cookbook and DVD: Tools and Techniques for Fighting Malicious Code (2011)

Chapter 11. De-obfuscation

De-obfuscation is the process of turning unintelligible information into something that you can understand. De-obfuscation is an art, a science, a hobby, and an undeniable requirement for malware analysis. This chapter classifies decoding, decryption, and packing as forms of obfuscation. Although these terms differ slightly in a technical sense, they’re all methods that attackers use to keep prying eyes off certain information. If you don’t learn de-obfuscation techniques, your understanding of malware and its capabilities will be limited. This chapter covers everything from reversing simple XOR routines to cracking domain-generation algorithms. You’ll learn how to decrypt command and control traffic and unpack binaries. As always, the best way to take your skills further after reading this chapter is to collect some malware (see Chapter 2) and practice, practice, practice!

Decoding Common Algorithms

XOR (exclusive-OR) and base64 encoding are two of the simplest and most common forms of obfuscation that you’re likely to run into. Most, if not all, programming languages, such as Python, C, Perl, JavaScript, PHP, Ruby, Delphi, and Visual Basic, support XOR and base64. Thus, the algorithms are simple to implement and convenient to access. The recipes in this section cover how to detect and decode data that has been obfuscated with XOR and base64.

Recipe 12-1: Reversing XOR Algorithms in Python

You can find supporting materials for this recipe on the companion DVD.

XOR is an example of a symmetric routine, which means the same key used to encode the data can be used to decode the data. Therefore, to reverse XOR, you need to know the initial value that attackers use when encoding the data. This recipe shows you how to decode various forms of XOR using a Python module called xortools.py that you can find on the book’s DVD.

Basic Properties of XOR

Table 12-1 shows how XOR operates. For each matching bit in the two operands, if both bits are the same, the result is 0; otherwise the result is 1. The ^ character represents an XOR operation in high-level languages such as C and Python.

Table 12-1: The Basic XOR Calculations

The special quality of XOR is that it reverses itself when applied to the same operand twice. For example, any time you XOR X with Y and then XOR the result with Y, you get the original value of X. Table 12-2 demonstrates this concept.

Table 12-2: Reversing XOR

Finding XOR in IDA Pro

If you have a copy of the malware that performs XOR operations, you can disassemble it with IDA Pro and look for XOR instructions. To do this, click Search⇒ text, and enter “XOR” into the input box. Don’t be surprised when you see hundreds of instructions such as XOR reg,reg (where reg is any general purpose register), because XOR-ing a value with itself will produce the value of zero. Therefore, many compilers use XOR reg,reg to represent statements like int i=0 in source code. You can safely ignore these instances of XOR, because they’re not what you’re looking for. Instead, look for instances of XOR that use hard-coded values or that reference memory addresses for the XOR key.

Single-byte XOR

Figure 12-1 shows a function that XORs 1 byte at a time using 0xBC as the key.

Figure 12-1: A function that performs single-byte XOR

According to the disassembly, the author of the program had something like the following as source code:

void xor_loop(unsigned char *pData)

{

for(int i=0; pData[i]; i++)

{

pData[i] ^= 0xBC;

}

return;

}

If you have a string, entire file, or bytes from a packet capture that attackers encoded with single-byte XOR, you can follow these steps to decode the data:

1. Copy the xortools.py file from the DVD that accompanies this book onto your computer. It contains the following function:

def single_byte_xor(buf, key):

out = ''

for i in buf:

out += chr(ord(i) ^ key)

return out

2. Now from a Python shell, you can do the following (assuming in_buf contains the encoded data):

$ python

>>> from xortools import single_byte_xor

>>> out_buf = single_byte_xor(in_buf, 0xBC)

>>> print out_buf

3. To XOR all bytes in the file input.txt with 0xBC and write the results to output.txt use the following:

$ python

>>> from xortools import single_byte_xor

>>> in_buf = open('input.txt','rb').read()

>>> out_buf = open('output.txt','wb')

>>> out_buf.write(single_byte_xor(in_buf, 0xBC))

>>> out_buf.close()

Four-byte XOR

Attackers commonly use XOR with a 4-byte key, because it provides stronger defense against analysts like you who are trying to decode the data. Instead of the 255 (0xFF) possible keys provided by 1-byte XOR, there are 4,294,967,295 (0xFFFFFFFF) possibilities. However, it’s all the same if you have a copy of the malware that encodes data and a few minutes to spare with IDA Pro. You’ll see an instruction such as XOR EAX,0x49171661 and then you’ve got the key.

The following function from xortools.py shows how you can use XOR with a 4-byte key.

def four_byte_xor(buf, key):

out = ''

for i in range(0,len(buf)/4):

c = struct.unpack("=I", buf[(i*4):(i*4)+4])[0]

c ^= key

out += struct.pack("=I", c)

return out

To use the code, follow the same steps as you did for 1-byte XOR, but call the four_byte_xor function instead:

$ python

>>> from xortools import four_byte_xor

>>> out_buf = four_byte_xor(in_buf, 0x49171661)

>>> print out_buf

Rolling XOR

Another implementation of XOR that you’ll run into is rolling XOR. In this case, the attacker supplies a sequence of bytes to use as the XOR key. The byte at offset 0 of the key is used to XOR the byte at offset 0 of the data to encode. The byte at offset 1 of the key is used to XOR the byte at offset 1 of the data, and so on…until the maximum length of the key is reached. At this time, the algorithm cycles back around to the beginning of the key and uses the byte at offset 0 to XOR the next byte in the data. Figure 12-2 shows an example of the algorithm used by the Limbo trojan to obfuscate stolen data before sending it across the network.

Figure 12-2: The rolling XOR key used by the Limbo Trojan

The following function from xortools.py shows how to implement a rolling XOR operation:

def rolling_xor(buf, key):

out = ''

k = 0

for i in buf:

if k == len(key):

k = 0

out += chr(ord(i) ^ ord(key[k]))

k += 1

return out

To decode Limbo’s stolen data using the key shown in Figure 12-2, you just need to do this:

$ python

>>> from xortools import rolling_xor

>>> out_buf = rolling_xor(in_buf, "canon75300USM")

>>> print out_buf

Brute-Force Guessing an XOR Key

If you don’t know the value that attackers initially used to XOR data, you can attempt to guess it using brute force. This method tries all possible XOR values (0 to 0xFF for 1-byte keys) on the encoded data until satisfying a specific condition. The conditions in this case are strings (or byte patterns) that you expect to find in the data once it’s properly decoded. You must at least have an idea of what to look for in the decoded file; otherwise, the algorithm won’t know when to stop.

Note We didn’t implement brute-force guessing of 4-byte XOR keys into xortools.py, because it’s too time-consuming. You could add this capability if you like, but compiling a program in C to perform the task might be quicker. In fact, Didier Stevens created a tool called XORSearch¹ that he wrote in C. XORSearch doesn’t support brute-force guessing on 4-byte keys either, but it does allow you to find patterns in ROL, ROR, and ROT encoded files.

The following function from xortools.py shows you how to implement brute-force guessing for 1-byte XOR keys. You pass it the encoded buffer, a list of strings that indicate success, and an optional start and end offset where the string must be found.

def single_byte_brute_xor(buf, plntxt, start=None, end=None):

for key in range (1,255):

out = ''

for i in buf:

out += chr(ord(i) ^ key)

for p in plntxt:

if out[start:end].find(p) != -1:

return (p, key, out)

return (None,None,None)

To perform a brute-force attack against the data in in_buf until the decoded buffer contains strings “http,” “www,” or “MZ” (a DOS header that indicates the beginning of an executable file), you could use the following code:

$ python

>>> from xortools import single_byte_brute_xor

>>> plaintext = ['http', 'www', '\x4d\x5a']

>>> (match, key, out_buf) = single_byte_brute_xor(in_buf, plaintext)

>>> if match:

>>> print 'Found a match for ' + match + ' using key ' + hex(key)

>>> print out_buf

When the single_byte_brute_xor function returns, it identifies which of the strings it finds in the decoded buffer, as well as the “winning” XOR key and a copy of the decoded buffer.

¹ http://blog.didierstevens.com/programs/xorsearch/

Recipe 12-2: Detecting XOR Encoded Data with yaratize

You can find supporting material for this recipe on the companion DVD.

This recipe shows you how to generate all 1-byte XOR permutations for a given string or sequence of bytes. You can then create a YARA rule from the resulting list and alert on any documents (PDF, DOC, SWF), packet captures, memory segments, and so on that contain an XOR-ed copy of your string. This is a great way to discover the XOR-encoded data without going through the process of manually inspecting everything that comes your way. The following code uses the single_byte_xor function from the previous recipe and is also integrated into xortools.py on the book’s DVD.

def get_xor_permutations(buf):

out = []

for key in range(1,255):

out.append(single_byte_xor(buf, key))

return out

def yaratize(rule, vals):

n = 0

strs = []

for val in vals:

s = ' $_%d { ' % n

for c in val:

s += "%2.2x " % ord(c)

s += '}'

strs.append(s)

n += 1

return """

rule %s

{

strings:

%s

condition:

any of them

}""" % (rule,'\n'.join(strs))

The following is an example of using the code to generate a YARA rule that detects any permutations of the string “This program cannot” (a substring of “This program cannot be run in DOS mode”). You’ll find this string in Windows binaries (e.g., EXE and DLL files), but you’re only looking for XOR-encoded versions of the string—which would show up only if someone intentionally tried to hide the string. You’re typing into a Python shell here and the command outputs a YARA rule.

$ python

>>> from xortools import get_xor_permutations as get_perms

>>> print yaratize('XorDos', get_perms("This program cannot"))

rule XorDos

{

strings:

$_1 = { 55 69 68 72 21 71 73 6e 66 73 60 6c 21 62 60 6f 6f 6e 75 }

$_2 = { 56 6a 6b 71 22 72 70 6d 65 70 63 6f 22 61 63 6c 6c 6d 76 }

$_3 = { 57 6b 6a 70 23 73 71 6c 64 71 62 6e 23 60 62 6d 6d 6c 77 }

$_4 = { 50 6c 6d 77 24 74 76 6b 63 76 65 69 24 67 65 6a 6a 6b 70 }

[REMOVED]

condition:

any of them

}

YARA was introduced in Chapter 3, so you should already be familiar with the rule syntax. YARA is fast, so you can generate large signature sets without any noticeable performance issues. The commands that follow create a rules file with all permutations of three different strings that we’d like to detect.

$ python

>>> from xortools import get_xor_permutations as get_perms

>>> rules = open('xorsigs.yar', 'w')

>>> rules.write(yaratize('XorDos', get_perms("This program cannot")))

>>> rules.write(yaratize('XorBank', get_perms('banking')))

>>> rules.write(yaratize('XorKernel', get_perms('kernel32.dll')))

>>> rules.close()

Now all you need to do is start looking for bad stuff:

$ yara -r -s xorsigs.yar Malware/

XorDos Malware/151147643

000006B5: FB C7 C6 DC 8F DF DD C0 C8 DD CE C2 8F CC CE C1 C1 C0 DB

XorDos Malware/29b01e816f0ba3735aeaa3517d653ccbc6342577.exe

0000046A: 2B 17 16 0C 5F 0F 0D 10 18 0D 1E 12 5F 1C 1E 11 11 10 0B

XorKernel Malware/7d927a57d0488f56e46f2073327bd1983b7e413d.exe

00005CF5: BD B3 A4 B8 B3 BA E5 E4 F8 B2 BA BA

XorDos Malware/8404200644217e86445d89d1f3ae8fee_oc.exe

00004BCC: 44 78 79 63 30 60 62 7F 77 62 71 7D 30 73 71 7E 7E 7F 64

XorKernel Malware/binaries/03d5fbb4bf2afca20dc78419abbe89f7

000E89E3: 9F 91 86 9A 91 98 C7 C6 DA 90 98 98

[REMOVED]

Immediately, this located a whole bunch of files that contain XOR-encoded executables. An equal number of files contain XOR-encoded versions of the string “kernel32.dll,” which you’ll frequently find in shellcode buffers.

Recipe 12-3: Decoding Base64 with Special Alphabets

Malware authors love base64 because it simplifies sending and receiving binary data over plain-text protocols. It’s very common to see malware making HTTP requests to URLs such as /page.php?v=dGVzdGluZw==, which is actually an attempt to exfiltrate binary data that’s been encoded with base64. This recipe shows how you can recognize and decode base64 data.

Recognizing base64 Data

The base64 algorithm translates each 3 bytes of binary data into four characters from the following 64-character set (known as the base64 alphabet):

ABCDEFGHIJKLMNOPQRSTUVWXYZ

abcdefghijklmnopqrstuvwxyz

0123456789+/

It is easy to visually spot base64 data because the string contains only those 64 characters. However, there is one exception—the 65^th character (=) is for padding. If the length of the data you want to encode is not a multiple of 4 bytes, the output will be padded. To recognize malware that uses base64, you can use the following YARA signature, which detects the presence of the base64 alphabet.

rule base64

{

strings:

//standard alphabet

$a="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"

//urlsafe alphabet

$b="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_"

condition:

$a or $b

}

Note The Perl script introduced in Recipe 10-10 detects base64-encoded strings in the Registry. It first checks if the length is an even multiple of 4 and then uses a regular expression (/[0-9a-zA-Z\+\/=]{$length}/) to validate the character set.

If you get any positive hits with this signature, you’ve probably found malware that uses base64. When you open up the file in IDA Pro, navigate to one of the string’s cross-references and you’ll find the base64 algorithm (see Figure 12-3 for an example). Of course, you don’t need to examine the algorithm in IDA to decode base64. However, malware commonly uses base64 in conjunction with an encryption algorithm. If you find the base64 function, you’re probably only a few steps away from the encryption algorithm.

Figure 12-3: Following the cross-reference to the alphabet leads you to the base64 algorithm.

Decoding base64 in Python

You can decode base64 data with the base64 and binascii Python modules. We are using the following POST request made by a Zlob² DNS changer variant for demonstration.

POST /index.php HTTP/1.1

Authorization: Basic

Content-Type: application/x-www-form-urlencoded

Content-Length: 74

Host: xx.255.186.237

Cache-Control: no-cache

x=MTkyLjE2OC4xMjguMTI4OzE5Mi4xNjguMTI4LjI7OzswOzE5

Mi4xNjguMTI4LjI1NDs7OzA=

As you can see, the POST payload appears to contain base64 data. All you need to do is paste that into a Python shell like this:

$ python

>>> import base64

>>> s = "MTkyLjE2OC4xMjguMTI4OzE5Mi4xNjguMTI4LjI7O

zswOzE5Mi4xNjguMTI4LjI1NDs7OzA="

>>> print base64.standard_b64decode(s)

192.168.128.128;192.168.128.2;;;0;192.168.128.254;;;0

After decoding, you’re left with the IP of the infected computer, the IP of its gateway router, the basic realm of the router, the username/password for the router (if Zlob was able to guess it), and the DHCP server’s IP. The standard_b64decode function decoded the input string using the standard 64-character alphabet that was presented earlier. However, not all base64 implementations use the standard alphabet. According to the RFC for base16, base32, and base64,³ there is no universally accepted alphabet. The / character isn’t safe in file names and URLs. The + and / characters are treated as word breaks by legacy text searching and indexing tools. Therefore, applications may choose a different alphabet. This is a problem because if malware encodes data with a non-standard alphabet, and then you try to decode it with the standard alphabet, you will not be successful.

Decoding with a Non-Standard Alphabet

urlsafe_b64decode decodes a string with a slight variation of the standard alphabet. It uses - instead of + and _ instead of / (the 63^rd and 64^th characters). You can call this function instead of standard_b64decode to automatically handle the character replacement. If you need to supply different values for the 63^rd and 64^th characters, you can do it with the b64decode function like this:

>>> decoded = base64.b64encode(the_string, ";]")

In most cases, you can survive using these decoding techniques. However, we have seen code that also uses a non-standard pad, such as . instead of the = character. You can use Python’s replacement method to translate the pad characters before decoding, like this:

>>> decoded = base64.b64encode(the_string.replace(".", "="), ";]")

The final situation we want to discuss is when malware authors try to be extra tricky and alter the ordering of the first 62 characters in the alphabet. For example, they may encode data using the base64 algorithm, but with the following character set:

ZYXWVUTSRQPONMLKJIHGFEDCBA

zyxwvutsrqponmlkjihgfedcba

9876543210_-

Notice how the ordering has all been reversed. Unfortunately, there’s no easy way to use Python’s base64 module for decoding this. The base64 module uses algorithms in binascii, which is built into Python. You would need to download the Python source code, modify Modules/binascii.c, recompile binascii.so, and then import the modified binascii module. So it’s possible, but not fun. A more practical suggestion is to use Google and find a C version of the base64 algorithm (search for base64.c or base64.cpp). Change the following lines as necessary, compile, and then you’ve got a custom base64 decoder.

static const char Base64[] = \

"ZYXWVUTSRQPONMLKJIHGFEDCBAzyxwvutsrqponmlkjihgfedcba9876543210_-";

static const char Pad64 = '.';

² http://www.faqs.org/rfcs/rfc3548.html

³ http://blog.washingtonpost.com/securityfix/2008/06/malware_silently_alters_wirele_1.html

Decryption

This section contains several recipes that are tied together to solve a common problem. In particular, you’re going to walk through the process of decrypting data that malware stole from a victim’s computer. You’ll likely never run into the same malware that we are using as an example, but it is representative of what you will find in the wild; and then you can use the same concepts to solve similar cases. In the scenario that’s described, imagine you’ve been supplied with a packet capture from the incident and a copy of the malware binary that allegedly produced the network traffic. Using these two resources alone and your investigation and reverse-engineering skills, you should be able to decrypt the data in the packet capture.

Recipe 12-4: Isolating Encrypted Data in Packet Captures

To begin, you should use a tool such as Wireshark to find the packets that contain encrypted data. Because you’re dealing with malware that steals information, you should focus on outbound packets first. Additionally, if the protocol is HTTP, it’s likely that the stolen data was transmitted in a POST payload. Once you’ve found the traffic, you can isolate the encrypted content from the rest of the packet capture. Figure 12-4 shows how you can export the POST payload from a packet capture and save it to a file on disk.

Figure 12-4: Exporting a POST payload with Wireshark

The data that you see in Figure 12-4 is encoded with base64. You can use the techniques described in Recipe 12-3 to decode the base64, but in this case, you’ll find that the data is still not readable. Consider the following commands (payload_base64.txt contains the extracted POST payload):

$ python

>>> import base64

>>> buf = open("payload_base64.txt").read()

>>> decoded = base64.standard_b64decode(buf)

>>> out = open("out.bin", "wb")

>>> out.write(decoded)

>>> out.close()

>>> exit()

$ xxd out.bin

0000000: 8bda 2d7f 2cac 67f3 ab04 a3ff 0cf2 e6f4 ..-.,.g.........

0000010: 30aa c7e8 fa7a e6e1 5966 cd30 ecbb 1eb5 0....z..Yf.0....

0000020: 1354 cd5f 5bd0 816f 9569 5a05 110b 640f .T._[..o.iZ...d.

0000030: 3e2b 5334 5b7d 2743 1e0e 7e9f 1373 e17e >+S4[}'C..~..s.~

[REMOVED]

As you can see, the out.bin file does not contain plain text. Thus, the malware must have encrypted the data in some way before encoding it with base64. The only chance you have at figuring out what type of encryption the malware used is to reverse-engineer a sample of the malware. That’s where you can find information about the encryption algorithm and encryption keys. When you first open the malware in IDA Pro and see 2000+ different functions, it can be a little discouraging. How in the world can you find the relevant code? A reasonable first step is to search the executable for calls to networking APIs because you know the malware sends encrypted data to a remote host. Figure 12-5 shows a decompilation (produced by IDA Pro’s Hex-Rays plug-in) of the function we found by following the cross-reference to HttpSendRequestA.

Figure 12-5: Locating the networking code in IDA Pro

As you can see, locating HttpSendRequestA landed us in a promising vicinity. The variable labeled v5 is the fourth parameter to HttpSendRequestA, which, if you look on MSDN,⁴ you will see is a pointer to any optional data to be sent immediately after the HTTP request headers. In other words, the v5 variable points to the POST payload—the encrypted data. If you examine how v5 is used before being passed to HttpSendRequestA, you can find the encryption code. Most likely, what you’re looking for is in one of the unlabeled subfunctions in the top of the image. We continue with our efforts in the next recipe.

⁴ http://msdn.microsoft.com/en-us/library/aa384247%28VS.85%29.aspx

Recipe 12-5: Finding Crypto with SnD Reverser Tool, FindCrypt, and Kanal

A time-saving trick you can use to quickly find encryption functions is to scan for cryptography constants or unique sequences of instructions used by cipher routines. You can use the following tools for this purpose:

· FindCrypt plug-in for IDA Pro:⁵ Copy findcrypt.plw to your plug-ins folder and then click Edit⇒ Plugins⇒ Find crypt. The following is an example of the results, which will show up in IDA’s output tab:

40C0F4: found sparse constants for MD4

42C244: found sparse constants for SHA-1

463F00: found const array Blowfish_p_init (used in Blowfish)

463F00: found sparse constants for HAVAL

463F20: found const array HAVAL_mc2 (used in HAVAL)

463F48: found const array Blowfish_s_init (used in Blowfish)

463FA0: found const array HAVAL_mc3 (used in HAVAL)

464020: found const array HAVAL_mc4 (used in HAVAL)

4640A0: found const array HAVAL_mc5 (used in HAVAL)

47A4B8: found const array SHA256_K (used in SHA256)

47A5E8: found const array SHA512_K (used in SHA512)

481800: found const array Rijndael_Te0 (used in Rijndael)

481C00: found const array Rijndael_Te1 (used in Rijndael)

482000: found const array Rijndael_Te2 (used in Rijndael)

482400: found const array Rijndael_Te3 (used in Rijndael)

482800: found const array Rijndael_Td0 (used in Rijndael)

482C00: found const array Rijndael_Td1 (used in Rijndael)

483000: found const array Rijndael_Td2 (used in Rijndael)

483400: found const array Rijndael_Td3 (used in Rijndael)

· SnD Reverser Tool:⁶ This application has a huge amount of hashing, conversion, and encryption-related functionality. Figure 12-6 shows an image of its output on our suspect binary. You can export results as a text file or as IDC, which you can then import into IDA for labeling.

Figure 12-6: Using SnD Reverser Tool to detect cryptography in a binary

· Krypto Analyzer plug-in for PEiD:⁷ In addition to scanning for cryptography constants, this tool also locates calls to encryption-related APIs such as CryptGenRandom, as shown in Figure 12-7. The Export button allows you to copy results to the clipboard or a text file, or export as IDC.

Figure 12-7: Using Kanal to detect cryptography in a binary

These cryptography-finding tools are most useful when they locate only a few constants. You can use IDA to navigate to the address identified in the tools and examine the cross-references to those functions. This should lead you straight to the code that handles the data to be encrypted and the encryption keys. Unfortunately, in the example case, the tools find so many different results that they don’t help you narrow the possibilities. Upon checking the binary for human-readable strings, you’ll find out why there is so much cryptography-related data in the malware. This sample has been static-linked with OpenSSL!

$ strings vendldr.exe

[REMOVED]

RSA part of OpenSSL 1.0.0a 1 Jun 2010

.\crypto\rsa\rsa_lib.c

X509_SIG

algor

RAND part of OpenSSL 1.0.0a 1 Jun 2010

@@.\crypto\rand\md_rand.c

When malware is static-linked with OpenSSL, a copy of the library’s code (including its functions, global variables, error messages, and so on) is compiled into the malware. OpenSSL is a large library (in particular libeay32.lib), so it increases the size of the malware significantly. However, there is good news. Because OpenSSL is so large, a majority of the functions in vendldr.exe probably belong to the library. Additionally, the strings output shows exactly which version of OpenSSL the attackers linked against (version 1.0.0a). The next recipe shows you how to use this information to reverse-engineer the encryption algorithm.

⁵ http://www.hexblog.com/2006/01/findcrypt.html

⁶ http://www.tuts4you.com/download.php?view.1923

⁷ http://www.peid.info/plugins/

Recipe 12-6: Porting OpenSSL Symbols with Zynamics BinDiff

There’s a very good chance that the unlabeled subfunctions in Figure 12-5 are calls to functions in the OpenSSL library. If you can figure out the names of the functions, you’ll be several steps closer to finding out which algorithms the malware uses. Because you know the version of OpenSSL, you can either compile libeay32.dll from source or download a precompiled copy⁸ of the DLL. Then, use a binary diffing tool to determine if the malware contains any of the same functions as libeay32.dll. This recipe uses Bindiff⁹ (see Recipe 3-11 for an introduction) to perform the analysis and then port the symbols (function names) and comments from libeay32.dll into the malware’s IDA database.

Porting Symbols with BinDiff

To compare two executables with BinDiff, follow these steps:

1. Create an IDA database (IDB) for both files.

2. Designate the malware (vendldr.exe) as the primary and libeay32.dll as the secondary.

3. Start with the primary IDB open in IDA and the secondary IDB closed. Then click Edit⇒ Plugins⇒ zynamics BinDiff 3.0 (or Shift+D).

4. Click Diff Database and select your secondary IDB.

Once the diff is complete, you have reached the turning point in the analysis of the malware’s encryption algorithm. This is when you go from the relatively clueless side to the well-informed side. As you can see in Figure 12-8, none of the functions in the primary IDB (the “name primary” column) have names, but the corresponding functions in the secondary IDB (the “name secondary” column) do have names. Highlight the functions whose names you want to import into the primary IDB (the authors selected all with a similarity and confidence >= 0.75) and right-click to select Port Symbols and Comments.

Figure 12-8: Porting function names into the malware’s IDA database

When you use IDA to navigate back to the function presented in Figure 12-5, you’ll see a drastic change. BinDiff labeled nine of the eleven unknown functions. We were able to use the OpenSSL API documentation¹⁰ to label the remaining two functions and assign meaningful names to the functions’ parameters. Figure 12-9 shows how the final product appears using the Hex-Rays decompiler.

From the Hex-Rays output, you can tell that the code creates an MD5 hash of the computer name and then uses it as the encryption key for Blowfish in CBC mode (indicated by the EVP_bf_cbc() function). The next recipe uses these details to figure out how to build a decryption tool in Python that can turn the data you found in the packet capture into plain text.

Figure 12-9: After porting symbols with BinDiff, you can see which OpenSSL functions are being called.

⁸ http://www.slproweb.com/products/Win32OpenSSL.html

⁹ http://www.zynamics.com/bindiff.html

¹⁰ http://www.openssl.org/docs/crypto/evp.html

Recipe 12-7: Decrypting Data in Python with PyCrypto

So far in this section of the chapter, you’ve isolated encrypted data from a packet capture, located the encryption functions in the malware’s binary, and labeled the IDA database accordingly. There is only a small amount of work left. In particular, you need to study OpenSSL’s EVP interface a bit more. The malware calls a function named EVP_EncryptInit_ex, so we found the definition for that function using the online documentation (see link in the previous recipe):

int EVP_EncryptInit_ex(

EVP_CIPHER_CTX *ctx, // an initialized cipher context

const EVP_CIPHER *type, // the cipher type

ENGINE *impl, // implementation (NULL == default)

unsigned char *key, // the symmetric key to use

unsigned char *iv); // the IV to use

Based on this information, you can tell that the second argument is the cipher type (Blowfish), the fourth argument is the key, and the fifth argument is the initialization vector. To summarize the information, the code displayed in Figure 12-9 does the following:

· Calls GetComputerNameA to query for the victim computer’s name.

· Computes an MD5 hash of the computer’s name and uses it as the encryption key for Blowfish in CBC mode.

· Uses an 8-byte IV for Blowfish that consists of the following values: 0B 16 21 2C 37 42 4D 58. These bytes are contained within a global variable in the binary that we found by tracing the fifth parameter (iv) to the EVP_EncryptInit_ex function.

· Encodes the encrypted data with base64 so that it can easily be transmitted over plain-text protocols.

· Sends the base64 string in the POST payload of an HTTP request.

Now you almost have all the required information to decrypt the data that you extracted from the packet capture. Because the symmetric encryption key is derived from the name of the victim computer, you need to know the name before you can attempt to decrypt the data. On a live machine, type echo %computername% at a command print to obtain the value that GetComputerNameAwould return. The name of the victim computer in this example is JASONRESACC69.

Decryption with PyCrypto

PyCrypto¹¹ supports the following algorithms:

· Hashing: MD2, MD4, MD5, RIPEMD, SHA1, and SHA256

· Ciphers: AES, ARC2, Blowfish, CAST, DES, DES3 (Triple DES), IDEA, and RC5

The following steps show how to install and use PyCrypto to decrypt the data in your packet capture:

1. Compile PyCrypto from source or type apt-get install python-crypto on an Ubuntu system. At last, it’s time to decrypt some data!

2. Pop into a Python shell and type the following commands to import the MD5 and Blowfish functions:

$ python

>>> import base64

>>> from Crypto.Hash import MD5

>>> from Crypto.Cipher import Blowfish

3. Decode the POST payload from the packet capture using the standard base64 alphabet:

>>> b64text = open("payload_base64.txt").read()

>>> decoded = base64.standard_b64decode(b64text)

4. Generate the MD5 hash for the infected computer’s name:

>>> md5 = MD5.new("JASONRESACC69")

5. Initialize a Blowfish object with the specified MD5 key, CBC mode, and the 8-byte IV:

>>> key = md5.digest()

>>> mode = Blowfish.MODE_CBC

>>> iv = "\x0B\x16\x21\x2C\x37\x42\x4D\x58"

>>> bf = Blowfish.new(key, mode, iv)

6. Complete the decryption and print the plain-text output:

>>> plaintext = bf.decrypt(decoded)

>>> print plaintext

ComputerName: JASONRESACC69

IP: 192.168.1.110

UserName: Jason

Country: US

Data: ltmpl=default&ltmplcache=2&continue=https%3A%2F%2F

mail.google.com%2Fmail%2F%3Fnsr%3D1&service=mail&r

m=false&ltmpl=default&ltmpl=default&Email=[REMOVED]

&Passwd=[REMOVED]&rmShown=1&signIn=Sign+in

URL: https://www.google.com/accounts/ServiceLoginAuth?service=mail

Title: Gmail: Email from Google - Microsoft Internet Explorer

There it is! The malware steals credentials from websites that users on the victim computer log into. This was a long, drawn-out process, but no one said it would be easy. Hopefully, you’ll experience the same warm, rewarding feeling that we do when you finally see the data that you worked so hard to decrypt.

¹¹ http://www.dlitz.net/software/pycrypto/

Unpacking Malware

If you try to statically analyze packed malware, you’ll notice an extreme shortage of information. You won’t find any interesting strings, the list of imported functions will be minimal, and all the program’s instructions will be encrypted. Your objective in unpacking is to remove the layer of obfuscation applied to the program when it was packed. There are many different methodologies for unpacking programs, most of which can be classified as manual or automated methods. Automated unpackers can definitely save you time, but you shouldn’t rely on them (they don’t always work) and you shouldn’t use them in lieu of learning the manual unpacking process. If you know how to manually unpack, you have knowledge to fall back on if your automated tools fail.

The following list shows the basic manual unpacking steps and the recipe number in this section where you can find more information. Throughout the section, the examples are based on unpacking variants of the Gozi (http://www.secureworks.com/research/threats/gozi/?threat=gozi) and Kraken (http://dvlabs.tippingpoint.com/blog/2008/04/28/kraken-botnet-infiltration) malware families. However, you can use the same tools and general guidelines for a majority of other malware. We chose these samples because the attackers obfuscated them with a custom packer as opposed to a well-known, publicly available one such as UPX, FSG, AsPack, and so on.

· Recipe 12-8: Finding OEP (the Original Entry Point). OEP is the address of the malware’s first instruction before it was packed.

· Recipe 12-8: Debugging the program until it reaches OEP. This allows the malware to execute far enough so that it unpacks itself in memory, but not so far that it begins executing the malicious code.

· Recipe 12-9: Dumping the unpacked process memory to a file on disk.

· Recipe 12-10: Rebuilding the Import Address Table (IAT) of the dumped file.

Before we begin, note that it’s not always possible to produce an exact duplicate of the original file when unpacking. But ask yourself—do you really need an exact duplicate? What problem are you trying to solve? If you want an unpacked copy of the file that you can execute on another machine, it will require significantly more work than if you just want to examine some of the unpacked file’s functions in IDA Pro.

Recipe 12-8: Finding OEP in Packed Malware

This recipe explains the concept of OEP and provides you with some techniques for finding OEP in packed malware. In most cases, you will notice that a file is packed when you open it in IDA Pro or when a packer detection utility (see Recipe 3-8) produces a positive match. Figure 12-10 shows how the packed sample of the Gozi trojan appears in IDA Pro. There are many heuristic indicators that the file is packed, such as the following:

· Small number of functions. The file only has eight built-in functions, whereas normal, unpacked programs will have many more.

· Small number of imports. The file imports fewer than 10 API functions from libraries supplied by the OS. This indicates that either the file is very limited in functionality or a packer has “hidden” the API functions.

· Large amount of unexplored space in the IDA color bar. The IDA color bar differentiates between the areas of a file that contain normal functions, data, and unexplored space. A large amount of unexplored space indicates that IDA cannot determine what those bytes in the file are used for (most likely because they’re encrypted).

· Encoding instructions inside a loop. The series of IMUL, ADD, SHR, XOR, and AND instructions with hard-coded numbers inside a loop indicates that the program performs some type of obfuscation or de-obfuscation.

Figure 12-10: Packed sample of Gozi loaded in IDA Pro

Finding OEP

Finding OEP can be simple or very challenging, depending on the packer. You’re essentially looking for a spot in the packed program where it has completed the decryption procedure. Using IDA Pro, you look for a location in the code that has an unclear destination (such as a jump or call to a dynamically determined location) or that doesn’t lead back inside the decryption loop like all the other instructions. Here are a few tricks you can use to try and locate OEP:

· The IDAGrapher¹² plug-in for IDA can help you by generating a graph with terminal blocks colored in green. A terminal block is a location in the code that returns or leads to an address assigned by a register or stack location. These blocks are likely candidates for transferring control to OEP.

· Try an automated, generic OEP finder listed in the Collaborate RCE Tool Library (see the links at the end of this section).

· Use a debugger and set a breakpoint on functions commonly called at the start of a program, such as GetVersionExA or GetCommandLineA (and the Unicode versions). If the program reaches one of these calls, the unpacking routine has likely finished. This won’t lead you to the exact instruction of OEP, but you’ll get close. It can also lead to some false positives if the unpacking routine (or a DLL loaded as a result of the unpacking routine) calls the APIs.

· Analyze the assembly code manually with IDA Pro and look for the terminal block. This method requires the most time and skill, but it’s also the most reliable once you get familiar with how to spot the right locations.

Figure 12-11 shows the location that we suspect transfers control to OEP. It happens to be inside the last subfunction called from the start function before the program terminates. All the other subfunctions appear to be helper routines for the unpacker. Thus, by the process of elimination, you can identify the instruction at 0x03000884. The malware must finish unpacking before reaching OEP and it must reach OEP before terminating. The location makes sense in the logical order of operations. It also fits because the CALL is leading to an unknown address (whatever is in EAX at the time) instead of a fixed location.

Figure 12-11: The CALL instruction that possibly leads to OEP

Reaching OEP in the Debugger

Once you’ve located an instruction that you believe leads to OEP, you have to execute the program until it reaches that instruction. As previously mentioned, if you stop executing too soon, the program won’t be finished unpacking. If you stop executing too late, the program will start to carry out the malware’s primary payload. In a debugger, you can set a breakpoint on the instruction’s address and let the malware execute until it reaches the breakpoint. This is what the authors did with Immunity Debugger, as you can see in Figure 12-12.

Figure 12-12: The malware paused in our debugger at address 0x03000884.

Notice how the EAX register contains 0x1AA061D0, which is where the CALL will lead. 0x1AA061D0 is very far from the base of the original program. In this case, you can assume that rather than decrypting instructions in place, Gozi allocates memory dynamically, performs the decryption on the new memory address, and then transfers control to them when finished. Now you can press F7 once in the debugger to “step into” the CALL, which takes you to 0x1AA061D0. Once you reach this point, scroll up and down in the debugger’s CPU pane and view the information presented in Figure 12-13. The IP addresses and hostnames for the command and control sites were not visible before unpacking the program. This is a good indication that you’ve reached OEP.

Figure 12-13: Visible strings in the program indicate you’ve reached OEP.

If all you want to do is debug the unpacked program, then you’re done. You’ve reached OEP and can begin to analyze the malware in a debugger. However, if you wish to extract a copy of the unpacked program for later analysis or for examination in IDA Pro, then proceed with the next recipes.

¹² http://dvlabs.tippingpoint.com/blog/2008/04/28/kraken-botnet-infiltration

Recipe 12-9: Dumping Process Memory with LordPE

This recipe picks up where Recipe 12-8 left off and shows how you can dump a copy of the unpacked process memory to disk once you’ve reached OEP. Before we begin, there are a few things you should know. You can use the tools and techniques described in this recipe on a majority of malware samples—not just the one used in the example. Also, you don’t have to find the exact OEP location to dump process memory–—you can acquire the dump at any time (assuming the malware doesn’t fight back against your memory dumping tools).

Process Dumping Tools

We have used the following tools with great success in the past:

· A standalone tool such as LordPE¹³

· A debugger plug-in such as OllyDump¹⁴

· A memory forensics platform such as Volatility (see Recipe 16-7)

The tool that you choose to use depends on how you are currently performing the analysis. A standalone tool such as LordPE can dump memory for any process on the system. A debugger plug-in such as OllyDump can only dump memory of a process that you are debugging. On the other hand, Volatility can extract the memory of a process from a RAM dump.

Using LordPE

Figure 12-14 shows the LordPE application. When you right-click a process to dump, you’ll see the options for dump full, dump partial, or dump region. If you choose dump full, LordPE will extract process memory starting at ImageBase and stopping at ImageBase+ImageSize. This is what you’ll typically choose, but as you can see in the figure, Gozi advertises an ImageSize of 0x1AA00000. That’s over 400MB, which is too large to be the real image size. It’s a simple anti-unpacking trick that causes LordPE’s dump full option to fail. If this happens, you can choose dump partial instead, and enter a valid value for the image size.

You may notice that LordPE’s menu displays the option “correct ImageSize,” but it’s not very reliable in our experience. You need to choose a value that is large enough to gather the whole unpacked program but small enough to not cause LordPE to access memory that isn’t allocated or that belongs to another module in the process. One way to get a more accurate size is to view the debugger’s memory map. Look for contiguous memory blocks starting at 0x1AA00000. In Figure 12-15, you can see three blocks, which total 0x31000 in size. Therefore, 0x31000 is what you should enter into LordPE’s size field for the dump.

Figure 12-14: Fixing the image size with LordPE before dumping memory

Figure 12-15: The debugger’s memory map shows which segments belong to gozi.exe.

At this point, you can save the output from LordPE to a file on disk and open it in IDA Pro. When you compare the Figure 12-10 (packed) with Figure 12-16 (unpacked), you’ll notice a significant change. In the unpacked version, you can see the entire list of functions and all of the imports, and—most important—the program’s instructions aren’t encrypted anymore.

Figure 12-16: The unpacked version of gozi.exe in IDA Pro

In this example of unpacking Gozi, we got lucky and only ran into one anti-unpacking trick (the invalid image size). The next recipe discusses a few roadblocks and how you can circumvent them.

¹³ http://www.woodmann.com/collaborative/tools/index.php/LordPE

¹⁴ http://www.woodmann.com/collaborative/tools/index.php/OllyDump

Recipe 12-10: Rebuilding Import Tables with ImpREC

You can find supporting material for this recipe on the companion DVD.

Many packers intentionally try to hide OEP by creating a spaghetti effect. This consists of hundreds of interwoven code blocks without an apparent end, and functions that never return (they just jump to another location). When you load a file packed with such a method into IDA, it just looks like a big maze and you could spend hours trying to find the instruction that leads to OEP. Figure 12-17 shows an example of how spaghetti packers appear. The following discussion uses the binary in this figure, which is a variant of the Kraken malware.

Figure 12-17: Kraken’s spaghetti packer prevents you from easily spotting OEP.

Combating the Spaghetti Packer

One of the tricks you can leverage, which is mentioned in Recipe 12-8, is loading the file in a debugger and setting a breakpoint on an API function frequently called at the start of a program. This method isn’t perfect because not all programs call the same functions during startup. However, if you’re using virtual machines to analyze the malware, it’s worth trying—and if your breakpoint doesn’t trigger, then you can just revert the machine and try something else.

In Figure 12-18, we loaded the “spaghetti-packed” binary in a debugger and set a breakpoint on GetCommandLineA (one of the functions commonly called from a program’s entry point). When the breakpoint is triggered, you can look in the stack pane and see that the return address is 0x0086F215. If a known module exists in this memory range, the module’s name is displayed next to the address—for example, modname.0086F215. Because there is no module name, no owner is associated with the address. This is very indicative of a packer that moves its code to an arbitrary memory segment, performs unpacking, and then resumes execution from the new address.

Figure 12-18: The stack pane reveals the caller’s address.

Using the debugger, you can navigate to 0x0086F215 in the CPU pane. This is the body of the unpacked malware. To dump the unpacked content to disk, you just need to know the base address and size of the memory segment that contains 0x0086F215. Recipe 12-8 showed how to use the debugger’s memory tab to investigate the base address and size, and then dump a range of memory with LordPE. We did the same thing for this example and found that the base address was 0x00860000 and the size was 0x18000. Then we loaded the dumped file into IDA Pro, however, it didn’t display as nicely as the Gozi sample. As you can see in Figure 12-19, many of the calls to API functions show addresses instead of the names. As a result, you can’t tell which functions are being called.

Figure 12-19: Calls to API functions are incorrect in IDA Pro.

Using Import REConstructor

Import REConstructor¹⁵ (ImpREC) is a tool you can use to rebuild the import tables of packed malware. The tool works by scanning the memory of a process for calls to imported functions. It builds a list of entries and then applies a patch to the file you dumped with LordPE. In particular, it modifies the PE header in such a way that it’s possible to determine which API functions are being called when you load the dumped file in IDA Pro.

When you start ImpREC, the tool gives you a list of processes from which you can choose. You’ll select the malware that you’ve got running in the debugger. Once you’ve done this, you can rebuild the process’s executable or one of its loaded DLLs. Now here’s the tricky part—for this spaghetti-packed malware, you don’t want to rebuild the process’s executable or any of its DLLs. You want to rebuild the module whose base address is 0x00860000. As you can see in Figure 12-20, after choosing our malware process, there is no option for rebuilding a module at 0x00860000.

So how do you use ImpREC to rebuild the import tables of a module that isn’t listed? First, you need to understand how ImpREC generates the list of modules that it does show. The tool reads the Process Environment Block (PEB) of the process and parses the InLoadOrder module list. These structures are discussed in detail in the beginning of Chapter 16, so you may want to quickly review that text. To trick ImpREC into “seeing” a module at 0x00860000, you can either add a module to the InLoadOrder module list or modify an existing module’s base and size. The trickimprec.py PyCommand (on the book’s DVD) for Immunity Debugger works using the latter technique. It modifies the base address and size of the process’s

Figure 12-20: ImpREC is not aware of the module at 0x00860000.

main executable (the one with a base address of 0x00400000 according to Figure 12-20) to a value that you specify. Here’s the code:

import immlib

import getopt

from string import atoi

def main (args):

imm = immlib.Debugger()

base = None

size = None

try:

opts, argo = getopt.getopt(args, "b:s:")

except getopt.GetoptError:

return "Usage: !trickimprec -b BASE -s SIZE"

for o,a in opts:

if o == "-b":

base = atoi(a, 16)

elif o == "-s":

size = atoi(a, 16)

if base==None or size==None:

return "Usage: !rebase -b BASE -s SIZE"

# pointer to PEB_LDR_DATA

ldr = imm.readLong(imm.getPEBaddress()+12)

# pointer to InLoadOrder list

load_order_list = imm.readLong(ldr+12)

# pointer to the first loaded module's base and size

# this will be to the exe image itself

ptr_base = load_order_list+24

ptr_size = load_order_list+32

mod_base = imm.readLong(ptr_base)

# overwrite the base and size with the values

# supplied by the user

imm.writeLong(ptr_base, base)

imm.writeLong(ptr_size, size)

You can use this plug-in by copying trickimprec.py from the book’s DVD into your PyCommands directory. Then type the following command in the debugger’s command box. For more information regarding PyCommands, see Recipe 11-8.

!trickimprec -b 0x00860000 -s 0x18000

Now, when you refresh ImpREC, it will think the wfsdmj.exe process exists at base address 0x00860000 instead of 0x00400000. Indeed, a copy of wfsdmj.exe exists at 0x00400000, but it’s the packed copy. The unpacked copy exists at 0x00860000 and that’s the one you want to rebuild. Notice in Figure 12-21 how ImpREC automatically recognized the new base address.

Getting the IAT Parameters

Regardless of whether you needed to take these extra steps to get the right module loaded in ImpREC, you’ll now need to tell ImpREC how to find the module’s import table. You can do this in an automated or manual manner. The automated method, which consists of clicking AutoSearch followed by Get Imports, is obviously the quickest, but it doesn’t always work. The manual method involves using your debugger to locate the import table and its size. You’ll enter the proper values into the RVA and Size field of ImpREC and click Get Imports.

To manually find the import table in your debugger, look for a call (any call) to an imported function in the unpacked malware. The GetCommandLineA identified earlier will do just fine. Right-click the instruction and choose Follow in Dump⇒ memory address. This will navigate to the memory address 0x00873060 in the dump pane. If you switch the format of the dump pane to Long/Address (this is all discussed in Recipe 11-3), then you’ll see the names of imported functions, as shown in Figure 12-22.

Figure 12-21: ImpREC can recognize the module at 0x00860000.

Figure 12-22: You can see the imported functions in the debugger’s dump pane.

Now you just need to scroll up in the dump pane until you find the start of the import table. It will be obvious because you’ll reach a point where there are no more function names. Also, scroll down to find the end of the import table. In this case, the start of the table was at 0x00873000 and the end was at 0x00873294. This gives you a size of 0x298 (0x294 plus 4 bytes for the final entry). Enter 0x298 in the ImpREC Size field. As for the RVA field, enter 0x00013000. The RVA (relative virtual address) is computed by subtracting the absolute address of the start of the import table from the base address of the module (0x00873000 – 0x00860000 = 0x0001300).

With the proper RVA and Size values filled into ImpREC, you can click FixDump. This launches a file browser where you can choose the file that you dumped with LordPE. ImpREC applies the patches and saves the changes to a new file named according to the original. For example, if your dumped file was C:\dumped.exe, ImpREC will create C:\dumped_.exe. Open the patched file in IDA Pro and you’ll notice how all of the imports have now been repaired. Figure 12-23 shows the final result. Notice that it’s the same code as shown in Figure 12-19, but with repaired import tables.

Figure 12-23: The repaired Kraken binary in IDA Pro

In this recipe, you learned how to circumvent several different challenges that you’ll likely encounter in the wild. First, you couldn’t find OEP due to the spaghetti packing. Then you had to patch some bytes in memory so that ImpREC could identify the module you wanted to rebuild. Finally, you manually located the import table and rebuilt the dumped file. Now you can analyze it freely in IDA Pro and see all of the function names.

Here are a few other tips and resources to keep in mind when manually unpacking malware:

· You only need to modify the OEP field in ImpREC if you use the AutoSearch feature or if you plan on re-running the rebuilt file on another machine. Otherwise, the entry point may be incorrect when you open the file in IDA Pro, but you’ll still be able to analyze all of the code.

· After you click Get Imports in ImpREC, you may notice a “valid:No” message beside some of the DLLs. If you expand the tree and view each of the imported functions in the DLL, you’ll notice that some are clearly invalid. Just right-click those entries and choose Cut Thunk to delete them, and the “valid:No” message will turn to “valid:Yes” when you’re done.

· The Universal Import Fixer (UIF) tool¹⁶ can automate the process of finding the import table, determining the table’s size, and performing various other IAT-related tasks.

· On Frank Boldewin’s website,¹⁷ he has posted at least three Flash tutorials on how to unpack malware using ImpREC, UIF, and OllyDbg.

¹⁵ http://www.woodmann.com/collaborative/tools/index.php/ImpREC

¹⁶ http://www.woodmann.com/collaborative/tools/index.php/Universal_Import_Fixer

¹⁷ http://www.reconstructer.org/papers.html

Unpacking Resources

Malware analysts and virus researchers have been dealing with packed code for many years. Sadly, we can’t cover more aspects of unpacking in this book, but we would like to point out some promising tools and concepts that you can use to further your knowledge. Table 12-3 shows a few of these resources and contains links to where you can find more information.

Table 12-3: Unpacking Resources

Debugger Scripting

This section describes how you can instrument malware samples using a debugger for the purposes of decoding or decrypting data. Michael Ligh and Greg Sinclair presented on this topic at Defcon16 (you can find the slides at http://mhl-malware-scripts.googlecode.com/files/Defcon2008_MalwareRCE_Ligh_Sinclair.pdf). The theory behind using a debugger to develop decryption utilities is that, as long as you can find the algorithm (i.e., decryption function) in a malware sample, you can execute the malware in such a way that you control the input to the function. Thus, if you have found encrypted data in a file or packet capture, you can stage that data in the memory of the malware process (using your debugger), supply it to the decryption function as an argument, and then capture the function’s output (the plain text data). In a sense, you are overriding the malware’s behaviors and default course of actions with your own.

The recipes that follow use Immunity Debugger’s Python interface to perform the instrumentation. However, you could just as easily use WinAppDbg (see Recipe 11-12) or IDAPython (http://code.google.com/p/idapython/).

Recipe 12-11: Cracking Domain Generation Algorithms

You can find supporting material for this recipe on the companion DVD.

If attackers hard-code the hostnames or IP addresses of their command and control (C2) servers into malware, it’s easier for the good guys to identify those machines and subsequently shut them down (by reporting inappropriate use to registrars or ISPs). Because this can put a major dent into a botnet’s operation, attackers started designing new ways for their malware to find C2 servers. One such alternative is known as a domain generation algorithm (DGA) and has been implemented into malware such as Kraken, Srizbi, Torpig, and Conficker. This recipe describes the concept of a DGA and shows how you can leverage debugger scripting to research the algorithms involved.

Domain Generation Algorithms

A DGA is an algorithm compiled into the malware’s executable that computes domains, given some value as input. You can think of this value as an encryption key or seed for the algorithm. Unless you know the seed and the algorithm, you can’t predict which domains the malware will contact. Early variants of the Conficker worm would generate a daily list of 250 domains based on the current date and try to contact each one. This resulted in the formation of the Conficker Working Group, a collaborative industry effort to combat Conficker by blocking access to each day’s list of domains. In response to this, new variants of Conficker were modified to generate 50,000 domain names a day. You can see how this can complicate efforts to block the miscreant’s access to the botnet. One would have to effectively prevent access to 50,000 new domain names every day.

Researching Kraken’s DGA

In the following tcpdump output, you can see a few of the domains generated by Kraken’s DGA.

$ tcpdump -r traffic.pcap -n dst port 53

reading from file traffic.pcap, link-type EN10MB (Ethernet)

IP 192.168.2.5.1025 > 4.2.2.1.53: 44608+ A? hmhxnupkc.mooo.com. (36)

IP 192.168.2.5.1025 > 4.2.2.1.53: 58435+ A? rffcteo.dyndns.org. (36)

IP 192.168.2.5.1025 > 4.2.2.1.53: 62018+ A? bdubefoeug.yi.org. (35)

This recipe shows you how to predict all the domains that Kraken’s DGA will generate, rather than just the small subset that you’ll get by executing the malware and capturing DNS lookups. To begin, you need a sample of the malware’s executable. Unpack it if necessary and navigate to the network-related APIs. In particular, look for calls to DnsQuery or gethostbyname because those are the APIs most programs use to resolve a domain to an IP. The following is pseudo-code based on what we saw in the Kraken sample:

unsigned int counter = 0;

char *pbuf;

struct hostent *rhost;

while(1) {

if (dga_get_domain(pbuf, counter++)) {

rhost = gethostbyname(pbuf);

if (rhost != NULL) {

if (try_connect(rhost)) {

break;

}

}

}

Sleep(1000);

}

The code shows Kraken calling a function named dga_get_domain within a while loop. During each iteration of the loop, the program increments the counter variable by one and passes that as the second parameter to dga_get_domain. The first parameter is an output buffer that receives the generated domain name. Believe it or not, this is 90 percent of what you need to know for cracking the algorithm. Indeed, other algorithms may be more complex, but Kraken’s is rather simple. It’s based entirely on the value of the counter variable. If you had access to Kraken’s source code, you could generate all possible domains using a loop like this:

counter = 0;

do {

dga_get_domain(pbuf, counter++);

printf("The domain is: %s\n", pbuf);

} while (counter < max_domains);

Wait a minute! Did we say “If you had access to Kraken’s source code”? Yes, we did, and while you probably don’t have the source code, if you have a copy of the malware (with the DGA algorithm compiled into it), then that’s good enough. Using a debugger, you can instrument the malware and make it repeatedly call dga_get_domain. Each time around, you’ll increment the value passed as its second parameter by modifying the stack of the running program. By setting a breakpoint at the end of dga_get_domain, you can tell when the algorithm is complete, and you can read the domain name from the output buffer.

Figure 12-24 demonstrates the logic behind this type of instrumentation. The chart on the left represents an uninstrumented program. It executes from start to finish as its author intended. The chart on the right represents an instrumented version. The debugger controls the program and only executes the function(s) required for generating the domains.

Figure 12-24: Example flow of execution for uninstrumented and instrumented malware

To instrument a program as we’ve described, here’s what you’ll need to know ahead of time:

· The DGA function’s starting address: Set the EIP register to this address before and after each iteration of the loop.

· The number and type of arguments that the DGA function accepts: Use this information to “fix” the stack so that the DGA function sees different arguments each time it executes.

· How to retrieve the DGA function’s return value: You need to know where to look (i.e., in a register, stack location, and so on) for extracting the generated domain.

The following code shows how you can implement the steps using a PyCommand for Immunity Debugger. When you call the program, you pass it the DGA function’s starting address. Everything else is done for you, including figuring out where the function ends, setting the breakpoints, incrementing the stack parameters, and reading the generated domains.

import immlib

import getopt

from string import atoi

def main (args):

imm = immlib.Debugger()

table = imm.createTable('Kraken Domains', ['Index', 'Name'])

dga_start = None

try:

opts, argo = getopt.getopt(args, "s:")

except getopt.GetoptError:

return "Usage: !kraken -s STARTADDR"

for o,a in opts:

if o == "-s":

dga_start = atoi(a, 16)

if dga_start==None:

return "Usage: !kraken -s STARTADDR"

func = imm.getFunction(dga_start)

imm.setBreakpoint(func.getEnd()[0]) # bp on the end

pbuf = imm.remoteVirtualAlloc(4) # for the output

for idx in range(0,100):

if idx % 2: continue # skip odds

# set EIP to the function's start

imm.setReg("EIP", dga_start)

# ESP+4 is the 1st argument and ESP+8 is the 2nd

imm.writeLong(imm.getRegs()['ESP']+4, pbuf)

imm.writeLong(imm.getRegs()['ESP']+8, idx)

# run until we hit a bp (the DGA function's end)

imm.Run()

# read the domain from the output buffer

host = imm.readString(imm.readLong(pbuf))

table.add('', ['%d' % idx, '%s' % host])

return "Done generating %d domains" % idx

The PyCommand creates a table with the generated domains, as you can see in Figure 12-25. When it’s done, you can copy the entire table (or just the names) and save them into a text file.

Figure 12-25: The debugger script outputs all of the generated domain names.

Note You might have noticed by looking at the index column in Figure 12-25 that the script iterates only from 0 (zero) to 100 and it skips the odd numbers. The counter variable is a 32-bit unsigned integer; thus it can range from zero to over four billion. There are two weaknesses in Kraken’s DGA that are worth mentioning:

· The counter starts at zero each time Kraken begins executing (i.e., every time an infected machine reboots) rather than at a random number between zero and four billion.

· Odd numbers cause Kraken’s algorithm to generate the same domain names as the even numbers that precede them. This effectively cuts the number of possible domains generated by the DGA in half.

As previously mentioned, the Kraken DGA is less complex compared to others. However, you can use the same concepts discussed in this recipe to try and crack them. The following are a few other resources you should look into if you’re interesting in DGAs:

· Downatool: Program and source code that implements Downadup.B/Conficker.B’s DGA (http://mnin.blogspot.com/2009/01/downatool-for-downadupbconflickerb.html)

· Conficker.C’s DGA: Reverse-engineered by SRI International (http://mtc.sri.com/Conficker/addendumC/)

· Technical details of Srizbi’s DGA: Reverse-engineered by Julia Wolf and Alex Lanstein of FireEye (http://blog.fireeye.com/research/2008/11/technical-details-of-srizbis-domain-generation-algorithm.html)

· “Taking over the Torpig Botnet” by Brett Stone-Gross and Marco Cova, et. al.: The document describes Torpig’s DGA, including how they used Twitter trends as a seeding mechanism (http://www.cs.ucsb.edu/~seclab/projects/torpig/)

Recipe 12-12: Decoding Strings with x86emu and Python

You can find supporting material for this recipe on the companion DVD.

In this recipe, we’ll show you how to reveal strings in a binary by using Chris Eagle’s x86emu¹⁸ plug-in for IDA Pro and Python scripting in Immunity Debugger. Most of the time, using the strings command (or BinText¹⁹ on Windows) is sufficient, but not always. If the binary is packed, you’ll need to unpack it first; and sometimes you’ll still find a shortage of visible strings. Even if you dump memory from a running process, you may not get a full list of the strings. The following pseudo-code demonstrates two of the reasons why gathering strings may not be so easy.

void do_work (void) {

// the string in encoded form

unsigned char str[] = "\x37\x11\x82\x75\x29";

// allocate a temporary buffer

char * tmp = (char *) malloc(sizeof(str));

// decode the string into the temporary buffer

decode(str, &tmp);

// use the string…

CreateMutex(NULL, NULL, tmp);

// zero-out the memory to erase the string

memset(tmp, 0, sizeof(str));

// free the temporary buffer

free(tmp);

}

if (condition) {

do_work();

}

You’ll run into two different issues with the previous code. Malware may decode a string immediately before using it and then zero-out or free the memory in which the string was stored before performing the next operation. Thus, only one string at a time is exposed in memory. The second issue is that the function that uses the string is only called if a certain condition is met. For example, if malware doesn’t receive a particular response from a command and control server, it may never call the do_work() function. In these cases, you’re not likely to find many strings in memory or in the unpacked file.

Finding SilentBanker’s Decoding Function

To demonstrate these concepts, we loaded a copy of SilentBanker into IDA Pro and navigated to the strings tab. As shown in Figure 12-26, even after we unpacked the binary, many of the strings are still unreadable.

Figure 12-26: Strings in the malware are unreadable even after unpacking.

We double-clicked one of the strings and then brought up the list of cross-references to the string by pressing Ctrl+X. This took us to the location in the SilentBanker’s code where the string is used. As you can see in Figure 12-27, the following steps are taken for each string:

1. A pointer to the string is moved into the EAX register.

2. The EAX register is pushed onto the stack twice (these become the function’s two arguments).

3. The sub_100122E8 function is called.

According to the usage, sub_100122E8 (presumably the decoding function) takes two arguments.

Figure 12-27: The function being called right after referencing the strings is probably the decoder.

The reason SilentBanker passes the same value twice to the decoding function was a bit puzzling at first. If both areguments are the same, wouldn’t it make more sense to create a function that only takes one argument? We came to the conclusion that sub_100122E8 is a generic function. It accepts a pointer to the input buffer (containing the data to decode) and a pointer to the output buffer (location to store the plain-text string). In the cases shown in Figure 12-27, the attackers are passing the same value twice, because they wish to decode the strings in-place. We can examine how the decoding function operates using x86emu.

Using x86emu to Investigate

The x86emu plug-in for IDA Pro allows you to execute instructions from the binary in an emulated environment. You can use it to investigate the behavior of certain code blocks without worrying about infecting your analysis machine. In IDA Pro, you can click on an address (0x10006897 in this case—where “NRRFSdxm Onre Hdlr” is moved into EAX) and then click Edit⇒Plugins⇒ x86 Emulator. This brings up the emulator’s control panel, with EIP automatically set to the location of the cursor, as shown in Figure 12-28. At this point, you can click 0x100068A3 (the first instruction after the call) and use the Run To Cursor button to execute the decoding function.

Figure 12-28: The x86emu control panel

If the instructions that you execute with x86emu modify data in the program, the changes are reflected immediately in the IDA database file (IDB). As shown in Figure 12-29, now we can see the newly decoded strings. We also labeled the sub_100122E8 function as Decode in the disassembly.

Figure 12-29: x86emu decoded the strings and automatically updated the IDA database.

If you just want to decode a few strings in the binary, x86emu is definitely the way to go. However, if you want to decode all strings, it could take some time. Remember, you can’t just emulate the entire program from start to finish, because some functions may not execute unless certain conditions are met. Instead, you could enumerate all cross-references to sub_100122E8function and force execution of each instance using a debugger script.

Forcefully Decoding All Strings with Python

By instrumenting code in a debugger, you can force the malware to decode all of its strings, without executing any of its malicious payloads. Here are the basic steps that the script takes:

1. It uses imm.getXrefFrom to enumerate all cross-references to the decoding function.

2. Starting at the address of the cross reference, it disassembles backwards (i.e., in a reverse direction) looking for the MOV r32,ADDR instruction, where r32 represents any 32-bit register and the ADDR operand is the address of the encoded string.

3. It reads a copy of the encoded string and saves it for logging purposes.

4. It sets EIP to the address of the cross-reference (the instruction which CALLs the decoding function), moves the string pointer onto the stack (twice—once for each argument), and uses imm.stepOver to execute the decoding function.

5. It reads a copy of the decoded string and prints it along with the encoded version saved in Step 3.

6. It repeats these steps for each string in the binary.

Here is the code:

import immlib

def main(args):

imm = immlib.Debugger()

table = imm.createTable('Silent Banker Strings',

['Address', 'Encoded', 'Decoded'])

# get all cross-references to the decoding function

refs = imm.getXrefFrom(0x100122E8)

for ref in refs:

addr = None

# disassemble backwards until finding MOV r32, <const>

for i in range (1,5):

op = imm.disasmBackward(ref[0], i)

instr = op.getDisasm()

if instr.startswith('MOV'):

# get address of the encoded string in memory

addr = op.getImmConst()

break

if addr != None:

# read the encoded version of the string

e_str = imm.readString(addr)

# forcefully execute the decoding of each string

imm.setReg('EIP', ref[0])

imm.writeLong(imm.getRegs()['ESP'], addr)

imm.writeLong(imm.getRegs()['ESP']+4, addr)

imm.stepOver()

# now read the decoded string

d_str = imm.readString(addr)

table.add('', ['0x%x' % addr, '%s' % e_str, '%s' % d_str])

To use the code, save it as a PyCommand and execute it with a copy of the malware loaded in Immunity Debugger. Keep in mind, the hard-coded address of the decoding function may be different between variants of the same malware. Figure 12-30 shows the output:

Figure 12-30: The output of our strings decoder plug-in

As you can see, the table shows the addresses of all strings, the encoded version, and the decoded version. Did you notice that we didn’t even look at the algorithm used in the sub_100122E8 function? It could be based on XOR, a simple substitution cipher, or a super complex formula. However, we were still able to decode all of the strings—that’s the power of instrumentation. As long as you can find the decoding function and learn 1) how it accepts input and 2) where it places the output, then you should be able to use similar techniques on other malware samples that you find in the wild.

¹⁸ http://www.idabook.com/x86emu/

¹⁹ http://www.foundstone.com/us/resources/proddesc/bintext.htm