Malware Analyst’s Cookbook and DVD: Tools and Techniques for Fighting Malicious Code (2011)

Chapter 4. Sandboxes and Multi-AV Scanners

Online sandboxes and multi-AV scanners can provide a quick and easy first impression of unknown files. In most cases, using these services requires little more effort than point, click, and read, but that is certainly not all you can do with them. Certain systems are designed to mask the back-end complexities and provide a very user-friendly and intuitive interface. Other systems are built to be flexible, allowing you to extend them with your own tools, scripts, and parameters. This chapter describes a few of the possibilities that can make your experience with sandboxes and multi-AV scanners even better.

Before we begin, you should understand the risks of using these services. False positives and false negatives will always be a problem. Even if 40 out of 40 antivirus products indicate that a file is safe, that doesn’t necessarily mean the file is safe. Additionally, unless you run a private instance of the service, the files you submit to public sites may be automatically shared with other vendors and third parties. This is generally good because the vendors need samples to build new signatures. However, targeted malware may contain hard-coded usernames, passwords, DNS names, or IP addresses of internal systems, which you don’t want distributed any more than necessary.

In addition to exposure of data to vendors and possibly the public, another factor to consider, that we previously described in Chapter 1, is notifying attackers that they’ve been detected. For example, if the attackers penetrated your network using a file with a specific MD5 hash, and two days later, a file with that hash shows up on a public scanner’s website, the attackers will know they’ve been detected. This may cause the attackers to change tactics or lay low until you think they’re out of your network.

Public Antivirus Scanners

Many antivirus vendors enable you to scan your entire computer free of charge on their websites using downloadable file scanners. However, few let you submit an individual file and get quick results. Even if they did allow the submission of a single file, why just get a single vendor’s results when you could get several? By using public antivirus scanners, you can go to a single website, submit a file, and have it quickly scanned by over three dozen antivirus products.

As previously mentioned, don’t take the results of a scan for granted. It is common for malware samples to remain undetected for hours, days, and even weeks after they’re released into the wild. The Race To Zero (http://www.racetozero.net) competition at Defcon 16 challenged researchers to modify ten viruses in a manner that allowed the viruses to retain their functionality, but be able to sneak by all major antivirus vendors. At least three teams completed the exercise in less than six hours! Malware authors play games as well. The group behind the Storm Worm used server-side polymorphic techniques, which resulted in minor changes to the malware’s code as frequent as every 10 minutes (see http://www.fortiguard.com/report/roundup_jan_2007.html).

Recipe 4-1: Scanning Files with VirusTotal

In the public antivirus scanner arena, VirusTotal¹ is the premier service. Its website allows you to upload suspicious files (sized 20MB or smaller) and scans them with 42 (the number at the time of this writing) antivirus products. You can use VirusTotal’s service in the following manner:

· Website submissions: The most common way to submit files is via the VirusTotal website. Navigate to the site, click the Browse button, and choose the file you want to upload. If you’re in a corporate environment and don’t want to trip any IDS or content-filtering alerts, you can choose to upload the file over an SSL connection.

· E-mail submissions: To submit files via e-mail, compose a new message to scan@virustotal.com, type “scan” in the subject field, and attach the file you want to have scanned. VirusTotal will return the results to you in an e-mail reply.

· Hash searching: VirusTotal’s website allows you to search their existing database of scanned files based on an MD5, SHA1, or SHA256 hash. This feature can be handy if you know a file’s hash value, but you don’t actually have a copy of the file.

· Explorer shell submissions: The VirusTotal Uploader is a Windows-only tool that allows users to upload files directly from Windows Explorer. You can download and install the tool by following the instructions at http://www.virustotal.com/advanced.html. Once installed, you can right-click on any file to send it to VirusTotal, as shown in Figure 4-1.

If the file you want to analyze is not already in the VirusTotal database, it will be uploaded. When the scan results are available, the uploader opens a browser on your machine to the VirusTotal web page so you can view them.

Figure 4-1: Submitting files with the VirusTotal uploader

Scan Results

The results page shows the antivirus product name, product version, date when the product’s signature definitions were last updated, and the detection name, if any. Figure 4-2 shows an example scan result.

Figure 4-2: VirusTotal’s scanning results page

As you can see, 29 out of 42 antivirus products detected the submitted file as malware and the other 13 reported that it was clean. The difference in results reinforces why scanning a file with multiple antivirus products is important.

In addition to antivirus results, VirusTotal provides information about the scanned file using various third-party tools and websites. The following list summarizes what you can find in this section of the results page:

· The file’s MD5, SHA1, SHA256, and ssdeep hash

· The file type (using TrID²)

· The file’s timestamp, entrypoint, sections, imports, and exports (using pefile³)

· A link to the ThreatExpert⁴ sandbox analysis (if one exists)

· A notice if the file’s digital signature is valid (using SigCheck⁵)

· A link to the Prevx⁶ analysis (if one exists)

· The name of any packers used to obfuscate the file (using PEiD)⁷

· A short description of the file if its hash is found in the National Software Reference Library (NSRL) Reference Data Set (RDS)

· A summary of the PDF tags using Didier Steven’s PDFiD⁸ (PDF files only)

A section of the VirusTotal output for the additional tools is shown in Figure 4-3.

Figure 4-3: VirusTotal’s extra information section

¹ http://www.virustotal.com

² http://mark0.net/soft-trid-e.html

³ http://code.google.com/p/pefile/

⁴ http://www.threatexpert.com

⁵ http://technet.microsoft.com/en-us/sysinternals/bb897441.aspx

⁶ http://www.prevx.com

⁷ http://www.peid.info

⁸ http://blog.didierstevens.com/programs/pdf-tools/

Recipe 4-2: Scanning Files with Jotti

Jotti’s malware scan⁹ is available in over ten languages and currently scans submitted files with 20 antivirus products. If a product is available for Linux, Jotti likely has it on its site. You can submit files to Jotti by using the web interface on the site’s home page.

Scan Results

The results page will show your queue status (if any) and then begin to update the page in real time with the results of each antivirus product. Jotti displays the date of the last virus definition update and text that displays either “Found nothing” in green or the name of the virus definition match in red. Figure 4-4 shows the appearance of Jotti’s results page.

Figure 4-4: Jotti’s malware scanning results

MD5 and SHA1 Hashes

Additionally, Jotti displays the MD5 and SHA1 hashes of the submitted file. You can search Jotti’s database by entering the MD5 or SHA1 hash into the following URL: http://virusscan.jotti.org/hashsearch.php.

⁹ http://virusscan.jotti.org

Recipe 4-3: Scanning Files with NoVirusThanks

The NoVirusThanks Multi-Engine Antivirus Scanner¹⁰ currently leverages 24 antivirus products to scan your submissions. You can use the NoVirusThanks service in the following manner:

· Website submissions (file upload): You can upload files sized 20MB or smaller to the NoVirusThanks website. An advantage to using NoVirusThanks is that you can request that the service does not distribute your files to other antivirus vendors and third parties. To do this, select the checkbox that says “Do not distribute this sample” when you upload your file.

· Website submissions (URLs): NoVirusThanks allows you to submit URLs. This means you do not need to download a potentially malicious file onto your computer first. To submit a URL, click the Scan Web Address tab, enter the URL, and click the Submit Address button. The NoVirusThanks system will grab the URL you submitted and begin to scan the file a short time later, just as if you had uploaded it directly.

· NoVirusThanks Uploader submissions: The NoVirusThanks Uploader¹¹ is a Windows-only application that allows you to upload files from your computer (5MB or smaller) without using a web browser. It also has an option to download files from a URL locally and then upload them. The application has a number of other features such as listing running services, automatic startup registry keys, loaded dynamic link libraries (DLLs), listing loaded drivers, and more. Figure 4-5 shows the NoVirusThanks Uploader application.

Figure 4-5: The NoVirusThanks Uploader application

Scan Results

Shortly after you’ve submitted files with any of the aforementioned methods, NoVirusThanks will assign a unique URL to your file. Note that this URL is unique per each upload, not each file. If you upload the same file on two separate occasions, you will receive two different URLs. The page displays the antivirus product name, the database or virus definition identifier, the antivirus engine version, and the detection name, if any, for each product. Figure 4-6 shows how the results appear.

Unfortunately, although NoVirusThanks provides the MD5 and SHA1 hashes for files on the results page, you have no way to go back to the website later and search for them. If you want to see a past file analysis, you must save the URL; otherwise, you must resubmit the file to obtain a new analysis for it.

Figure 4-6: NoVirusThanks scanning results page

NoVirusThanks offers a few other products and services that you may be interested in as well. Here are some short descriptions:

· Threat Killer is a scriptable malware remover that you can use to unload drivers, terminate processes, delete files, and delete registry keys. The fact that it is scriptable is nice, because sometimes to remove malware effectively, you need to do things in a particular order. Antivirus programs may be hard-coded to perform actions in a specific order, causing them to fail.

· Hijack Hunter is a tool that scans for common indications of infection, such as changes to the HOSTS file, Browser Helper Objects, DNS servers, and registry startup locations.

· URLVoid is an online service that you can use to check if a given domain is malicious based on results from Google Diagnostic, McAfee SiteAdvisor, Norton SafeWeb, and others (17 in total, currently).

¹⁰ http://scanner.novirusthanks.org

¹¹ http://www.novirusthanks.org/products/

Recipe 4-4: Database-Enabled Multi-AV Uploader in Python

You can find supporting material for this recipe on the companion DVD.

This recipe presents a command-line interface to VirusTotal, Jotti, and NoVirusThanks. The script gives you the ability to analyze files using multiple services without using a web browser or a special client. Since it is written in Python, it works on Linux, Mac OS X, and Windows. You must not use this script for commercial purposes or in manner that violates the vendor’s acceptable use policy.

With the ability to upload files on the command line, you can easily automate submissions and retrieve the results. For example, you could create a second script to extract potentially dangerous attachments from a local MBOX file or from a remote POP3/IMAP account; then pass the attachments to avsubmit.py. You could link this script into your honeypot workflow, as described in Chapter 2, or use it to automatically submit processes that you dump from memory with Volatility. The possibilities are endless.

Here is the usage for avsubmit.py:

$ python avsubmit.py –h

Usage: avsubmit.py [options]

Options:

-h, --help show this help message and exit

-i, --init initialize virus.db

-o, --overwrite overwrite existing DB entry

-f FILENAME, --file=FILENAME

upload FILENAME

-v, --virustotal use VirusTotal

-e, --threatexpert use ThreatExpert

-j, --jotti use Jotti

-n, --novirus use NoVirusThanks

Usage: avsubmit.py [options]

If you call avsubmit.py once with the --init flag, it creates an empty file named virus.db (a SQLite database). Each time you use the script in the future, it automatically populates the database with the antivirus scanning results. If you don’t want to use SQLite for tracking your analysis, just don’t initialize the database.

Submissions to VirusTotal

You can upload files to VirusTotal by specifying the -v flag. The avsubmit.py script computes the hash of your input file and checks VirusTotal’s hash search to see if there are already results for the file. If so, the script queries for the list of detections. Otherwise, the script uploads your file, waits for the processing to complete, and then returns the list of detections. Before using the script, you must obtain a VirusTotal API key¹² and paste it into the top of avsubmit.py.

$ python avsubmit.py -f 11229.exe –v

Using VirusTotal...

Searching VT for SHA1: 590933753cac80734db00c5e5d7f8063bcc1e4d5

The file does not already exist on VT

Submitting file to VT, please wait...

Analysis here: http://www.virustotal.com/analisis/\

cec813ceaa070d1e0fadd8ea09e58f88445d0950999d8e4948d8c104b9b94a5f-1269588142

Trying to get results for the next 600 seconds...

Prevx => High Risk Worm

NOD32 => a variant of Win32/Kryptik.DHB

F-Prot => W32/Alureon.H.gen!Eldorado

Symantec => Suspicious.Insight

McAfee+Artemis => Artemis!C178CBB6E88D

Sophos => Mal/TDSSPack-W

CAT-QuickHeal => Win32.Packed.TDSS.z.5

Authentium => W32/Alureon.H.gen!Eldorado

VirusBuster => Rootkit.Alureon.Gen.10

TrendMicro => TROJ_BREDO.SME

Submissions to Jotti

If you specify the –j flag, then the script checks if your file is already in Jotti’s database. If not, it performs the submission. You’ll receive the list of detections on the command line, as well as a URL to the results page.

$ python avsubmit.py -f 11229.exe –j

Using Jotti...

Initialized cookie: sessionid=ced321e4eca5aad8940055dc51cd193a4

Initialized APC: 8f0b8b63d15375760b14c195419d6369a5d92564

Checking Jotti for MD5: C178CBB6E88DFA8AFEB1E2F740EBF72B

Analysis here:

http://virusscan.jotti.org/en/scanresult/\

c9738bd6346142b20df79091f1b741098a90116b

Trying to get results for the next 60 seconds...

nod32 => Win32/Kryptik.DHB

fsecure => Packed:W32/TDSS.EU

avast => Win32:Malware-gen

gdata => Gen:Heur.Krypt.25

kaspersky => Packed.Win32.TDSS.z

asquared => Packed.Win32.TDSS.z!A2

avira => TR/PCK.Tdss.Z.3138

ikarus => Packed.Win32.Tdss

avg => Agent_r.RG

sophos => Mal/TDSSPack-W

quickheal => Win32.Packed.TDSS.z.5

virusbuster => Rootkit.Alureon.Gen.10

Submissions to NoVirusThanks

NoVirusThanks does not support searching for files by hash, so avsubmit.py always uploads your file without first checking if it’s previously been submitted. It will wait for the scanners to complete, print results to STDOUT, and provide a link where you can find the analysis in a browser.

$ python avsubmit.py 11229.exe -n

Using NoVirusThanks...

Submitting file to NoVirusThanks, please wait...

http://scanner.novirusthanks.org/analysis/c178cbb6e88dfa8afeb1e2f740ebf[REMOVED]

NOD32 => Win32/Kryptik.DHB

a-squared => Packed.Win32.Tdss!IK

TrendMicro => TROJ_BREDO.SME

VBA32 => BScope.Rootkit-Dropper.TDSL

Dr.Web => BackDoor.Tdss.based.5

Avast => Win32:Alureon-FW [Rtk]

Avira AntiVir => TR/PCK.Tdss.Z.3138

Kaspersky => Packed.Win32.TDSS.z

BitDefender => Gen:Heur.Krypt.25

Ikarus T3 => Packed.Win32.Tdss

Panda => Trj/TDSS.EF

G-Data => Packed.Win32.TDSS.z

AVG => Agent_r.RG

F-PROT6 => W32/Alureon.H.gen!Eldorado

Comodo => TrojWare.Win32.Trojan.Agent.Gen

Querying the virus.db Database

Once you have processed a few samples, you can begin to execute queries on your virus.db database. The SQLite API is available for many languages including PHP, Perl, Python, and C, so with just a few lines of code you could generate useful trends and statistics about your malware collection. For the following example, we’re just using the command-line sqlite3 client to query for any Rustock samples in the database.

$ sqlite3 virus.db

SQLite version 3.5.9

Enter ".help" for instructions

sqlite> .schema

CREATE TABLE detects (

id INTEGER PRIMARY KEY,

sid INTEGER,

vendor TEXT,

name TEXT

);

CREATE TABLE samples (

id INTEGER PRIMARY KEY,

md5 TEXT

);

sqlite> SELECT t1.md5,t2.vendor,t2.name

...> FROM samples AS t1, detects AS t2

...> WHERE t2.name LIKE "%Rustock%" AND t1.id=t2.sid;

00bd6c02dcdb4bf8f8545ca47e8f3c16|VirusBuster|Backdoor.Rustock.EQ

00bd6c02dcdb4bf8f8545ca47e8f3c16|Microsoft|Backdoor:Win32/Rustock.E

0f543e220474bb41cc4b47e2cce6162d|Microsoft|Backdoor:Win32/Rustock.E

sqlite>

Here are a few additional notes about the avsubmit.py script:

· If you want to use all supported services at once, specify –jevn as a parameter.

· You can import avsubmit.py from your own Python scripts, which would enable you to format the output any way you want. In fact, the script in Recipe 8-7 works in this described manner. Here is an example of how to import the VirusTotal class from another Python script:

from avsubmit import VirusTotal

vt = VirusTotal(sys.argv[1]) # first argument is a file name

detects = vt.submit()

for key,val in detects.items():

print " %s => %s" % (key, val)

¹² http://www.virustotal.com/advanced.html

Multi-Antivirus Scanner Comparison

It’s always good to have options, and that’s just what you get with the various multi-AV scanning services. If nothing else, multiple services can come in handy if one of the other scanning services is down or under a heavy load. Table 4-1 compares some key features, options, and attributes of the profiled online antivirus scanning services. You can use the information to determine which service is best for your goals. Of course, the data can and will change in the future, so keep that in mind.

Table 4-1: Antivirus Scanner Comparison

Public Sandbox Analysis

Public sandboxes execute malware in a monitored environment so that you don’t have to risk infecting your own machines to perform behavior analysis. Sandboxes record changes to the file system, registry keys, and incoming/outgoing network traffic, then make the results available to you in a standardized report format. In the next few recipes, we’ll discuss a few of the common sandboxes that you can leverage for a quick analysis of potentially malicious files.

Recipe 4-5: Analyzing Malware with ThreatExpert

The ThreatExpert¹³ advanced threat analysis system (ATAS) executes files in a virtual environment and reports the changes made to the file system, registry, memory, and network. According to its website, ThreatExpert works by taking snapshots of the system before and after executing the malware in order to determine what changed, in addition to using API hooks that intercept the malware’s interactions in real time. You can expect to find the following information in a ThreatExpert report:

· Newly created processes, files, registry keys, and mutexes

· Contacted hostnames or IP addresses, along with hex and ASCII dumps of the network traffic

· Virus-scanning results for the submitted file and any created files

· Possible country of origin, based on heuristic factors such as geographical location of an IP the file contacts or traces of foreign languages found in the file

· Categorization (such as backdoor or keylogger) along with a relative severity level

· Screenshots from the analysis if a new window is detected

You can submit files (up to 5MB in size) to ThreatExpert by using their web form. Submissions require an e-mail address, and in addition to showing the results online, ThreatExpert will e-mail you a copy of the report files in a Zip archive. An alternate tool that you can use for uploading is the ThreatExpert Submission Applet,¹⁴ which is a Windows-only GUI application for submitting files.

Figure 4-7 and Figure 4-8 show example content from a ThreatExpert report.

ThreatExpert users also have the option to register for an account and login prior to submitting. By doing so, all submissions from a particular account (e-mail address), even those made through the Submission Applet, will be linked together. Users can view or execute searches against their previous submissions.

Figure 4-7: ThreatExpert’s summary and technical details (truncated)

Figure 4-8: ThreatExpert’s country of origin and network traffic results

¹³ http://www.threatexpert.com/submit.aspx

¹⁴ http://www.threatexpert.com/submissionapplet.aspx

Recipe 4-6: Analyzing Malware with CWSandbox

CWSandbox was designed by researchers at the University of Mannheim in Germany. Sunbelt Software licensed the sandbox code for commercial purposes, so you can purchase your own installation of CWSandbox and customize it as you desire. However, both the University of Mannheim and Sunbelt Software still offer publicly accessible (and free) interfaces to submit malware for analysis. To submit code to one of the free sandboxes, you can visit http://www.sunbeltsecurity.com/sandbox/ or http://mwanalysis.org.

CWSandbox works by injecting DLLs into newly created processes. The DLLs hook Windows API functions in order to spy on the malware’s behavior as it executes. The website warns that malware can bypass the hooks by calling native API functions directly or by making calls from kernel mode. Despite this limitation, CWSandbox is still very effective for most malware. Here are a few differences between the free and commercial versions of CWSandbox:

· You can submit Windows PE (portable executable) files to Sunbelt’s free interface. The commercial version lets you submit URLs, BHOs, zipped files, or infected documents.

· You can submit files to the free sandbox via a web browser. The commercial version lets you submit files via e-mail, nepenthes honeypots, or a local directory on the server’s file system.

· The commercial version lets you control the target system on which the malware runs. For example, you could use VMware or a standalone non-virtual system.

· The commercial version includes a behavior summary based on detections such as downloading PE files from the Internet, creating files in the system32 directory, or injecting code into other processes.

As shown in Figure 4-9, CWSandbox shows detailed results on a per process basis. This is very valuable for malware that drops multiple executables, and you want to know which component is responsible for creating a particular file, registry key, or other artifact on the system. If you’re using a sandbox solely based on a diff between before and after snapshots, you will not receive this type of granular information.

Figure 4-9: CWSandbox lists the changes made by each process

Recipe 4-7: Analyzing Malware with Anubis

Anubis¹⁵ is a sandbox for analyzing unknown binaries. Unlike CWSandbox, Anubis is privately owned and operated and is not available for sale (as far as we know). When you submit files to Anubis, you can use the default form or an advanced submission form. The following list outlines some of the possibilities provided by Anubis:

· If you submit a URL instead of a file, Anubis opens the URL in Internet Explorer, essentially turning the sandbox into a client honeypot. This is very useful if you are aware of a suspect website or file on the Internet and you want to validate the behavior of a system when visiting that URL.

· You can upload auxiliary files in addition to an executable. Anubis provides this capability because some executables require companion files (such as configurations or DLLs) to execute properly. Alternately, you can upload all files using a Zip archive (non-password protected or protected with the password “infected”).

· You can download reports in HTML, XML, plain text, or PDF formats, as well as a full packet capture.

· You can submit samples to Anubis over an SSL channel by changing http:// in the URL to https://.

Figure 4-10 shows the analysis results for a file submitted to Anubis. In the created files section, you can see that v2captcha.exe created captcha.dll and captcha.bat.

Figure 4-10: Anubis results can help you quickly identify the malware family

In the Processes Created section, you can see that the malware executed the batch file by passing it on the command line to cmd.exe. If you search online for the created files, you’ll see that they are components of the Koobface worm. In particular, we found the Malicious Social Networking: Koobface Worm¹⁶ article by Joel Yonts that helped us correlate the findings.

¹⁵ http://anubis.iseclab.org/

¹⁶ http://www.sans.org/security-resources/malwarefaq/koobface-worm.php

Recipe 4-8: Writing AutoIT Scripts for Joebox

You can find supporting material for this recipe on the companion DVD.

Joebox,¹⁷ by Stefan Buehlmann, is a sandbox designed with flexibility and customization. You can submit files to Joebox using the web interface, or you can contact Joe Security for information about purchasing your own instance. An advantage to using Joebox is that the system uses SSDT and EAT hooking in the kernel to monitor the malware’s behavior, as opposed to hooking Windows API functions in user mode like other sandboxes. As a result, the Joebox analysis loses a small amount of high-level context (such as if a new process were launched with ShellExecute or WinExec). However, it greatly reduces the chance that malware could bypass the monitors by calling native APIs in user mode or by directly calling the kernel mode function from a loaded driver. Here are some additional features of Joebox:

· Joebox supports analysis of executables, DLLs, kernel drivers, Word documents, PDFs, and more.

· You can choose to execute your malware on Windows XP (the default), Windows Vista, and/or Windows 7.

· You can set up Joebox to execute malware on a non-virtual and non-emulated system. Joebox uses the FOG imaging solution¹⁸ (also see Recipe 7-7) to revert systems back to their original state after every infection.

· You can acquire full packet captures for the malware you submit, allowing you to analyze the network traffic using a tool of your choice.

· You can download modules for the amun and nepenthes honeypots to automatically submit new malware samples to Joebox.

· You can write scripts in the AutoIT¹⁹ language to customize the environment in which your malware executes.

Note The JoeBox website does not maintain an online copy of the malware analysis. You must keep the analysis you received in e-mail if you want to access it at a future date. Otherwise, you must resubmit the file to receive a new analysis.

Writing Scripts for Joebox

Joebox scripts are text files with a .jbs extension. You can write them using any text editor, or SciTE4AutoIt3 (the AutoIT editor). The Joebox website provides a few sample scripts and some documentation about their API. The following is a short description of the scripts that are currently available:

· Simulate user interactions to click through an installer (a component of many fake antivirus programs).

· Scrape a web page for URLs and visit them each sequentially in a browser (essentially a lightweight web crawler).

· Compute behavior diffs to reduce the amount of noise involved in standard API monitor logs. For example, you can record the activities made by IE when visiting a legitimate URL, then record the activities when IE visits a malicious URL, and report the differences.

The following is an example of a bare Joebox script:

Script

; choose Windows XP

_JBSetSystem("xp")

; start the analysis

_JBStartAnalysis()

; start the sniffer

_JBStartSniffer()

; execute the uploaded malware

_JBLoadProvidedBin()

; let the malware run for 120 seconds

Sleep(120)

; stop the sniffer

_JBStopSniffer()

; stop the analysis

_JBStopAnalysis()

EndScript

The script selects Windows XP as the target environment by passing xp to _JBSetSystem. You can optionally replace xp with vista (for Windows Vista) or w7 (for Windows 7). Then it starts the analysis, starts the network sniffer, executes the malware that you uploaded along with the script, and lets the malware run for 120 seconds. The total time of your script cannot exceed four minutes on the public Joebox systems. Figure 4-11 shows the Joebox submission form where you would choose the malware file and script to upload.

Figure 4-11: Submitting scripts to Joebox

The next few recipes describe a number of ways you can turn the bare Joebox script into extremely useful tools.

¹⁷ http://www.joebox.org/submit.php

¹⁸ http://www.fogproject.org/

¹⁹ http://www.autoitscript.com/wiki

Recipe 4-9: Defeating Path-dependent Malware with Joebox

You can find supporting material for this recipe on the companion DVD.

In some cases, malware will simply terminate if it is not executing from a particular location, such as the system directory (C:\WINDOWS\system32 on XP). Because you cannot control the location on disk where sandboxes place your files before executing them, the file will likely fail to run. Of course, this will lead to the sandbox not showing any results, which may lead you to believe that the file is non-malicious. In this recipe, we’ll show you how to use a Joebox script to copy a file to a given directory before executing it. First, consider the following source code, which is an example of malware that performs a path check before infecting a machine.

int main(int argc, char* argv[])

{

char sysdir[MAX_PATH];

char modulename[MAX_PATH];

GetSystemDirectoryA(sysdir, MAX_PATH);

GetModuleFileNameA(NULL, modulename, MAX_PATH);

// exit if not in the system32 directory

if (strstr(modulename, sysdir) == NULL) {

ExitProcess(0);

} else {

//Infect the system!

}

return 0;

}

You can use the following Joebox script to copy your malware into the system directory and then launch it.

Script

_JBSetSystem("xp")

_JBStartAnalysis()

_JBStartSniffer()

; copy the submitted file to system directory

$NewFile = @SystemDir & "/" & "malware.exe"

FileCopy("c:\malware.exe", $NewFile, 1)

; execute the file from its new path

Run($NewFile, @TempDir, @SW_HIDE)

Sleep(120)

_JBStopSniffer()

_JBStopAnalysis()

EndScript

The script begins by selecting XP as the operating system and starting the analysis and sniffer. Next, it uses the AutoIT language to copy the C:\malware.exe file (your uploaded submission) into the system directory. Once the copy is complete, it runs the file. This is all you need to execute path-dependent malware in an automated sandbox.

Note Many sandboxes place uploaded files in a specific location (such as C:\malware.exe). One of the ways malware can detect that it is running within a sandbox is by checking for the existence of those hard-coded file names. How you can bypass that? Easy. Upload a Joebox script that copies C:\malware.exe to another path such as C:\betya\wontguessthis.exe, delete the original C:\malware.exe, and then run the malware.

Recipe 4-10: Defeating Process-dependent DLLs with Joebox

You can find supporting material for this recipe on the companion DVD.

Many sandboxes are capable of launching DLLs, but they use generic host processes such as rundll32.exe or custom programs that call LoadLibrary. As you will learn in Chapter 13, DLLs often check the name of their parent process and only exhibit certain behaviors if inside a particular process. In this recipe, we’ll show you how to use a Joebox script to analyze a DLL inside one or more host processes of your choosing.

Using the following Joebox script, you can get your DLL loaded into Internet Explorer.

Script

; access to the IE-related functions

#include <IE.au3>

_JBSetSystem("xp")

_JBStartAnalysis()

_JBStartSniffer()

; copy the submitted file to system directory

$NewFile = @SystemDir & "/" & "malware.dll"

FileCopy("c:\malware.dll", $NewFile, 1)

; add the AppInit_DLLs entry

RegWrite(

"HKLM\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Windows",

"AppInit_DLLs", "REG_SZ", "malware.dll")

; browse to this site in IE

$oIE = _IECreate("http://banksite.com")

Sleep(120)

; done with IE now

_IEQuit ($oIE)

_JBStopSniffer()

_JBStopAnalysis()

EndScript

The script works by registering the DLL in the AppInit_DLLs registry key and then creating a new instance of Internet Explorer. The new IE process will automatically load malware.dll. If the DLL needs to be registered as a Browser Helper Object instead (BHO), it’s just a matter of entering the right registry keys before launching IE.

In a similar scenario, you may need to load a DLL into Explorer; however, AppInit_DLLs only takes effect for new processes. One of the ways you can do this, albeit quite messy, is to terminate the explorer.exe process. If Explorer ever crashes, winlogon.exe will automatically re-start it, which is when your AppInit_DLLs entry will load. The following script contains the necessary code for the described method.

Script

Func KillProcess($process)

Local $hproc

Local $pid = ProcessExists($process)

If $pid = 0 Then

Return

EndIf

$hproc = DllCall(

"kernel32.dll", "hwnd", "OpenProcess",

"dword", BitOR(0x0400,0x0004,0x0001),

"int", 0, "dword", $pid)

If UBound($hproc) > 0 Then

If $hproc[0] = 0 Then Return

Else

Return

EndIf

$hproc = $hproc[0]

Local $code = DllStructCreate("dword")

$ret = DllCall(

"kernel32.dll", "int", "TerminateProcess",

"hwnd", $hproc, "uint", DllStructGetData($code,1))

Return

EndFunc

_JBSetSystem("xp")

_JBStartAnalysis()

_JBStartSniffer()

; copy the malware

$NewFile = @SystemDir & "/" & "malware.dll"

FileCopy("c:\malware.dll", $NewFile, 1)

; add the AppInit_DLLs entry

RegWrite(

"HKLM\SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Windows",

"AppInit_DLLs", "REG_SZ", "malware.dll")

; terminate the process so it restarts

KillProcess("explorer.exe")

Sleep(10000)

_JBStopSniffer()

_JBStopAnalysis()

EndScript

The script defines a local function named KillProcess, which uses DllCall (an AutoIT API) to call OpenProcess and TerminateProcess. You can use DllCall in your AutoIT scripts to locate and invoke any Windows API functions. Thus, you have the power to configure the sandbox in very specific ways before executing the malware.

Recipe 4-11: Setting an Active HTTP Proxy with Joebox

You can find supporting material for this recipe on the companion DVD.

In this recipe, we assume you want to analyze malware that makes an outbound HTTP connection to an attacker-controlled server. The server responds differently to IP addresses in different countries, and you want to elicit a particular response by sending your request from a specific country. The first part is up to you—find open HTTP proxies hosted in your target country, or acquire a cheap virtual server hosted in the target country and set up your own HTTP proxy. You can learn exactly how to do this by reading Recipe 1-4.

Then you can use the following Joebox script to configure the proxy:

Script

_JBSetSystem("xp")

_JBStartAnalysis()

_JBStartSniffer()

; identify your proxy server IP and port

$ProxyServer = "1.2.3.4:8080"

; alter the machine's proxy settings

RegWrite(

"HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings",

"ProxyServer", "REG_SZ", $ProxyServer)

RegWrite(

"HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings",

"ProxyEnable", "REG_DWORD", 1)

_JBLoadProvidedBin()

Sleep(10000)

_JBStopSniffer()

_JBStopAnalysis()

EndScript

As long as the malware uses derivatives of the WinINet API functions, your proxy configuration will work. In particular, the malware must call InternetOpen with the INTERNET_OPEN_TYPE_PRECONFIG flag, which causes the application to look up proxy configuration from the registry. If the malware uses the Urlmon API (UrlDownloadToFile) or implements its own HTTP handlers using Winsock (send and recv), then your proxy configuration will not work. This is just an example of the type of control that you can exercise over the target system by using Joebox scripts.

Recipe 4-12: Scanning for Artifacts with Sandbox Results

You can find supporting material for this recipe on the companion DVD.

Online sandboxes have massive databases that display file names, registry keys, mutexes, and other artifacts created by malware. In most cases, you can determine if the same or similar malware ran on a system that you’re investigating by checking for the existence of such artifacts. Given the ability to collect the artifacts of samples analyzed by online sandboxes, you could create a lightweight artifact database for detecting related infections.

The dbmgr.py and artifactscanner.py scripts on the DVD are examples of a generic, reusable scanning framework. The examples in this recipe show how to enumerate artifacts from ThreatExpert reports. You can populate your collection manually or write additional modules for other online sandboxes. The basic idea is to start with a SQLite database schema that describes all the data you want to collect (files, registry keys, and so on). Then you can write plug-ins that collect those artifacts from various sources and insert them into your database. When it’s time to perform an investigation, you can quickly check if the target system is infected by any malware that you have previously analyzed.

Managing the Artifact Database

The following output shows the syntax for dbmgr.py, an interface for adding, deleting, and querying data in your artifact database.

$ python dbmgr.py -h

Usage: dbmgr.py [options]

Options:

-h, --help show this help message and exit

-i, --init initialize DB

-s, --show show entries in DB

-a ADD, --add=ADD add md5 to DB

-d DELETE, --del=DELETE

delete md5 from DB

-b PAGE, --bulk=PAGE bulk import page

The first step you should take is to initialize a new artifact database. You can do that by passing the --init flag, like this:

$ python dbmgr.py --init

Success.

$ ls -al artifacts.db

-rw-r--r-- 1 root root 5120 2010-04-04 20:42 artifacts.db

You should now have a file named artifacts.db in your current working directory, built with the following schema:

CREATE TABLE samples (

id INTEGER PRIMARY KEY, // unique id of each sample

md5 TEXT // md5 hash of sample

);

CREATE TABLE files (

id INTEGER PRIMARY KEY,

sid INTEGER, // corresponds to samples.id

filename TEXT, // path to new file on sandbox

md5 TEXT // md5 of newly created file

);

CREATE TABLE mutants (

id INTEGER PRIMARY KEY,

sid INTEGER, // corresponds to samples.id

name TEXT // name of new mutex on sandbox

);

CREATE TABLE regkeys (

id INTEGER PRIMARY KEY,

sid INTEGER, // corresponds to samples.id

keyname TEXT, // registry key name

valuename TEXT, // newly created value under keyname (if any)

data BLOB // data for newly created value (if any)

);

The samples table contains columns with an MD5 hash of all malware in your database, along with an auto-incrementing unique ID for each sample. The files, mutants, and regkeys tables all have a column named sid, which corresponds to the unique ID of the malware sample that created the artifact. To add artifacts from an existing ThreatExpert report, you can pass the sample’s MD5hash and the --add flag, like this:

$ python dbmgr.py --add=0xD289CD91759850640B8C260EDC651D51

Checking ThreatExpert for MD5: D289CD91759850640B8C2[REMOVED]

Analysis: www.threatexpert.com/report.aspx?md5=D289C[REMOVED]

Added sample with ID 1

[FILE] a5bc910a81a305994[REMOVED] %AppData%\BifroXx\server.exe

[FILE] a5bc910a81a305994[REMOVED] %ProgramFiles%\BifroXx\server.exe

[MUTEX] Bif1234

[REGKEY] HKEY_LOCAL_MACHINE\SOFTWARE\

Microsoft\Active Setup\

Installed Components\{9D71D88C-C598-4935-C5D1-43AA4DB90836}

[REGKEY] HKEY_LOCAL_MACHINE\SOFTWARE\BifroXx

[REGKEY] HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\

Control\MediaResources\msvideo

[REGKEY] HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\

Control\MediaResources\msvideo

[REGKEY] HKEY_CURRENT_USER\Software\BifroXx

The dbmgr.py script imports the ThreatExpert class from the avsubmit.py module (see Recipe 4-4) to get access to the HTML returned by ThreatExpert’s website for a given file. In total, the script added eight artifacts (five registry keys, two files, and one mutex) to the database.

You can add the most recent 20 reports on ThreatExpert by using the --bulk=1 flag. Each time you increment the integer, it grabs the next most recent 20 reports.

$ python dbmgr.py --bulk=1

Checking ThreatExpert for MD5: dada441f3cd70903433c71fb63fe4ae4

Analysis: www.threatexpert.com/report.aspx?md5=dada441f[REMOVED]

Added sample with ID 2

Checking ThreatExpert for MD5: 91481733[REMOVED]

Analysis: www.threatexpert.com/report.aspx?md5=91481733[REMOVED]

Added sample with ID 3

[FILE] c54f8ceb7c792f8fe2231d8b40ad780b %Temp%\RarSFX0\CleanNV.exe

[FILE] 0679a1ebaf691168a25961eb50cf3fdc %Temp%\RarSFX0\CleanTool.exe

[FILE] 3221d42b5ebf1e505396dcc9e8527f0a %Temp%\RarSFX0\CTREBOOT.exe

[FILE] c93ab037a8c792d5f8a1a9fc88a7c7c5 %Temp%\RarSFX0\NeroCheck.exe

[REMOVED]

Note The artifact database is similar in concept to an antivirus signature database; thus, its results are subject to false positives and false negatives. Be extra careful when using the bulk import, because it automatically adds artifacts to your database. If someone uploads a legitimate file, such as iexplore.exe (Internet Explorer) to ThreatExpert and then you gather the artifacts and scan for them on a machine, you’ll end up detecting IE rather than malicious code.

Once you have added samples and artifacts to your database, you can print the contents before using it. To do this, pass the --show flag. The output shows the ID for each sample, its MD5 hash, and the list of files, registry keys, and mutexes associated with the sample.

$ python dbmgr.py --show

ID MD5 Hash

------------------------------------------------------------

1 D289CD91759850640B8C260EDC651D51

[FILE] a5bc910a81a3059[REMOVED] %AppData%\BifroXx\server.exe

[FILE] a5bc910a81a3059[REMOVED] %ProgramFiles%\BifroXx\server.exe

[REGKEY] HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\

Active Setup\Installed Components\

{9D71D88C-C598-4935-C5D1-43AA4DB90836}

[REGKEY] HKEY_LOCAL_MACHINE\SOFTWARE\BifroXx

[REGKEY] HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\

Control\MediaResources\msvideo

[REGKEY] HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\

Control\MediaResources\msvideo

[REGKEY] HKEY_CURRENT_USER\Software\BifroXx

[MUTEX] Bif1234

2 dada441f3cd70903433c71fb63fe4ae4

3 91481733005406e14439eb78308e7aa7

[FILE] c54f8ceb7c792[REMOVED] %Temp%\RarSFX0\CleanNV.exe

[FILE] 0679a1ebaf691[REMOVED] %Temp%\RarSFX0\CleanTool.exe

[FILE] 3221d42b5ebf1[REMOVED] %Temp%\RarSFX0\CTREBOOT.exe

[FILE] c93ab037a8c79[REMOVED] %Temp%\RarSFX0\NeroCheck.exe

[REMOVED]

Management with SQLite Database Browser

The SQLite Database Browser²⁰ provides a GUI front end for working with SQLite databases. Thus, if you’re not familiar with SQL, you can still add, remove, or modify artifacts. You can install it on Ubuntu by typing the following command:

$ apt-get install sqlitebrowser

You can also download binaries from the tool’s website to run it on Windows or Mac OS X. Once you have the tool installed, launch it like this:

$ sqlitebrowser artifacts.db

Figure 4-12 shows the tool’s GUI.

Figure 4-12: Viewing artifacts with SQLite Database Browser

Scanning for Infections with Your Artifacts.db

The final step in this recipe is to take your artifacts.db and use it to detect artifacts on the potentially infected system. In the following example, we use a script called artifactscanner.py, which is a Python script designed to execute on a live Windows machine. If the target system does not have Python installed, you could compile artifactscanner.py with py2exe or write a similar program in C using the SQLite C API. However, even in that case, rootkits that hide files and registry keys could cause artifactscanner.py to report incorrect results. A more forensically sound method is to acquire disk and memory images and then use the artifacts database in one (or more) of the following manners:

· Write a plug-in for The Sleuth Kit (see Recipe 10-2) that scans a hard drive mounted read-only for files in your database.

· Write a RegRipper plug-in (see Recipe 10-8) that scans hive files for registry keys in your database.

· Write a Volatility plug-in (see Recipe 17-11) that scans a memory dump for the mutexes in your database.

When executing artifactscanner.py, you can scan for one type of artifact at a time by passing --files, --regkeys, or --mutants. Alternately, you can scan for all types of artifacts by passing their short names like –frm. The only modifier for scans is the --strict flag, which is applicable during file scans. A strict scan produces alerts only when it finds a file on the suspect media with the same full path as a file in the database and matching MD5 hashes as well. Otherwise, the script uses loose mode, which produces alerts on any files with the same full path, regardless of the MD5 hash.

Figure 4-13 shows an example of the artifact scanner in action.

Figure 4-13: The artifact scanner found traces of three different malware infections

The scanner detected infections from three unique malware samples, based on information in the artifact database. It identified files named herss.exe, captcha.dll, and winlogin.exe in specific paths where previous malware samples dropped files with the same names. Furthermore, it detected a suspicious mutex named with the CAPTCHA prefix, which is similar to captcha.dll. If you recall from Recipe 4-7, a Koobface variant created catpcha.dll. Therefore, it is very likely that Koobface also created the mutex. If the artifact scanner detects the presence of the same mutex on another machine in the future, you will automatically know it is infected.

For each of the artifacts, the tool prints a link to the original source of information (ThreatExpert) so you can look up additional details on the malware that may be present on the suspect machine.

²⁰ http://sqlitebrowser.sourceforge.net/