Digital Archaeology (2014)

9. Document Analysis

One of the great challenges for a digital investigator comes when the evidence is in plain sight but can’t be found—or isn’t recognized for what it is. Perhaps a file isn’t what it says it is. On the simplest level, a JPEG image file might be renamed with an AVI extension, making it appear to be a video file. More complex techniques employed by the bad guys include embedding files within files (alternate data streams) or even burying small files in the Windows Registry. This chapter covers some of these techniques and how to uncover the evidence.

File Identification

In theory, the easiest aspect of a file search is the process of identifying what kind of file it is. The Windows file systems (and less universally, other file systems as well) use the file extension as a file identifier. A file with the name of IMAGE.JPG is an image file using the Joint Photographic Expert Group file compression algorithm. Usually. However, file systems do not enforce extension rules. If a user changes the extension of a file in Windows, the only action the operating system takes is to present the warning shown in Figure 9.1.

Figure 9.1 Changing a file extension in Windows only prompts a warning.

Once the extension is changed, the Windows default actions for the newly assigned extension will apply to the file. An example of a default action is the double-click of the mouse. In Windows, a double-click opens the file in the application associated with that file type. A file with a .doc extension opens in Microsoft Word. When the extension is changed, double-clicking on the file in Windows Explorer will not elicit the expected behavior. In the example shown in Figure 9.2, the file CyberControls Forensic Approach.pdf has been renamed to CyberControls Forensic Approach.tif. Double-clicking on the file in Explorer brought up the Office Picture Manager application, which could not open a PDF file. So Windows displayed a small square with a red X instead of the PDF. Right-clicking on the file and selecting Open With, and then selecting Adobe Reader opens the file correctly.

Figure 9.2 Windows default actions do not behave as expected when a file extension has been changed.

What does this little exercise imply? For one thing, it becomes clear that changing the extension does nothing to alter the structure of the file. It is doubtful anyone thought it would. Additionally, changing the extension did not affect the ability of the correct host application to open the file. Therefore, there is something else in the file structure that determines whether or not an application can open a file.

File Structure

Any file stored by any file system used by a computer must have similar structure or there can be no cross-platform compatibility. However, anyone who has experience using more than one OS knows that a Microsoft Word document can be opened with either Word for Macintosh or Word for Windows. Additionally, third-party applications such as OpenOffice have no trouble opening the files of either version of Word.

The extension, discussed briefly in the previous section, acts as a superficial identifier of file type. If unaltered, most applications have no difficulty identifying files by extension. If asked to open a file with an unfamiliar extension, the application will dig a little deeper into the file. There are several internal identifiers that files use to introduce themselves. These include file metadata and file structure.

File Metadata

Files can contain two types of metadata that applications use to recognize and open the file. Internal metadata is contained within the file and can consist of a binary string or a text string. Three commonly used metadata containers are the MFT attributes, the file header, and the magic number.

MFT Attributes

The NTFS file system used by current Windows OSs places certain information concerning the file into the metadata files. Chapter 8, “Finding Lost Files,” contains a table of the metadata files used by NTFS, along with a description of each file. The metadata files are created when the disk is first formatted to NTFS. After that, every file copied to the drive owns at least one MFT record. MFT records contain certain attributes that define the entry to the OS. Table 9.1 lists the MFT attributes used by NTFS, along with the hexadecimal value assigned to each attribute.

Table 9.1 NTFS Attributes

The reason for understanding the various attributes is that some of the utilities used key on those attributes when they search for file data. An important aspect to remember is that file content is considered an attribute of the file. It is the data attribute and carries the hex value of 0x80. This value is useful to utilities such as Directory Snoop in that it allows the application to identify the part of the file that is specifically user data. Figure 9.3 illustrates how Directory Snoop reads the file attribute types.

Figure 9.3 Not all files contain internal metadata.

Additionally, MFT records can be examined to see if a file that no longer exists perhaps once did. While the Windows metafiles are not directly viewable by most applications, a disk editor such as Directory Snoop allows this task to be accomplished. In Figure 9.4, we see the entry for a file called redstateicon.png. The file no longer exists on my hard disk, but a string search of the $MFT metafile in Directory Snoop found this record. While most of the information is gibberish to the average person, directory Snoop does tell us the file name and what application is the default program to open the file. Additionally, it tells us what the legacy MS-DOS file name is as well.

Figure 9.4 This Notepad file is cut and pasted from a copy of the MFT record found in the NTFS $MFT metafile.

File Header

File headers get their name as a result of being the first string of data read by the file as it loads. In Chapter 8, I discussed the concept of file headers and EOF markers. Applications read these bytes when first asked to load the file. If the header does not correspond to the file type identified, the application may have difficulty loading the file. To the digital investigator, the header also provides the starting point for carving files out of slack space or unallocated space on storage media. Table 9.2 lists a few examples of some common file types, their extension, header, and, where applicable, EOF marker. It is a very small sample—to attempt including every file type would fill an entire book.

Table 9.2 Sample File Structure and Metadata

A general rule of thumb (not universally applied by all developers) is that a humanly readable file has humanly readable headers and EOF markers. A binary file has binary metadata. Notice in Table 9.2 that the PDF file has plain text metadata. An ASCII text file has no internal metadata, as seen in the Directory Snoop screenshot in Figure 9.3.

Magic Numbers

Magic numbers are really nothing more than another method of structuring a header. Files incorporating magic numbers embed a file signature consisting of hexadecimal code into the first few bytes of the file to identify the file type. The term is derived from the Linux and UNIX (*nix) file system. The Linux Information Project (LIP 2006) defines the magic number as occupying the first six bytes of the file. Many programs use the magic number as the first step in identifying a file type. However, as with the file header, there are certain files, such as ASCII text files, HTML, and source code cabinets, that do not incorporate magic numbers.

Identifying a file by the magic number method does incorporate a small degree of latency (additional processing overhead required by an application to perform a specific set of tasks). Most Linux builds have defined lists of magic numbers in various directories. Among these are (Darwin 1999)

• /usr/share/file/magic.mgc—Compiled list of magic numbers

• /usr/share/file/magic—Default list of magic numbers

• /usr/share/file/magic.mime.mgc—Default compiled list that will display mime types when the -i trigger is used

• /usr/share/file/magic.mime—Default list that will display mime types when the -i trigger is used

In Linux, the file(1) command can be used to identify a file by its type. One of the command’s first tests is to attempt to read a magic number and compare the number it finds to one or more of the magic number lists above. The digital forensic examiner can use a disk editor to view a file and examine the magic number directly in an effort to identify the file type.

Understanding Metadata

The word metadata gets thrown around a lot and is used in more than one context. Earlier in this book, a loose definition of metadata was presented that simply defined it as data that describes data. However, metadata can exist in multiple forms. The operating system maintains information about files in various repositories. As discussed in the previous chapter, the NTFS file system makes use of a series of metadata files. Individual files can also contain information stored within the file that defines the file. Additionally, many applications, such as document management systems, maintain separate files containing metadata. All of these sources can be a gold mine of information for an investigator. Aguilar v. Immigration and Customs Enforcement (2008) determined that the three types of metadata relevant to digital evidence include

• System metadata—Information generated by the file system or document management system

• Substantive metadata—Information that defines modifications to a document

• Embedded metadata—Information embedded by the application that creates or edits the file

Substantive metadata can fall within either of the other categories. Another form of metadata that exists that is important to the investigator is external metadata. Many document and image management software solutions maintain large amounts of information in the form of a database. Indexing, file modification, tracking, and auditing information is stored in separate files maintained by the application. Each of these types of metadata will be discussed over the next few pages.

System Metadata

Chapter 8 introduced the concept of metadata usage by the OS. All file systems maintain vast amounts of information about the files and directories stored on the volumes they control. The fact is—the file system is the metadata that the operating system uses to manage files on the various media it controls. To be certain, there are physical aspects of the file system, such as the mapping of file allocation units on the drive itself, but that mapping is meaningless without the directions that tell the OS or the applications how to get there from here.

It isn’t just hard disks that have volumes of metadata. CD-ROMs, DVDs, and even thumb drives need some form of file table that informs the system how and where files are store. Every computer running needs to be able to mount multiple file systems. The hard disk uses its system. As mentioned in the previous chapter, Linux systems might be formatted with the Ext2, Ext3, or Ext4 or perhaps the Reiser file system. Windows typically uses NTFS, although some legacy systems may use one of the several versions of FAT. Even an NTFS-based computer needs to be able to read FAT if that is how a USB flash drive was formatted. And in order to read CD-ROMs or DVDs, the ISO-9660 or the ECMA-167 file system must be mounted. Understanding how these files systems work is far beyond the scope of this book. However, a brief overview of how and where the system metadata is stored is essential for the digital investigator.

Value of OS Metadata

The useful aspect of OS metadata in the process of digital investigation is the ability to prove the existence of a deleted document and to research the timeline of a document. OS metadata does not help identify contents of files, aside from file type. A critical piece of information found here is the modify/access/create (MAC) data. Disks formatted with NTFS offer the additional attribute of entry modified (EM). EM notes the last time the MFT entry in the NTFS metafiles was modified. MAC information is valuable for creating a time line of events, as long as care is taken in analyzing and interpreting the data. It is important that the tools used by a forensic investigator are tested and verified to not alter MAC data.

All files stored on any file system are stamped with the time and date they were created, the last time they were accessed, and the last time they were modified. MAC data is easily viewed using a wide variety of commercial and shareware utilities. Figure 9.5 shows an example. Used in conjunction with other information found on the computer, it might be possible to identify what user was the last to access or modify a file and perhaps even who created it. A short discussion about each of the MAC attributes is in order here.

Figure 9.5 Several readily available utilities allow the user to view the currently active MAC data for a file.

Create

The create attribute on a file is generated the first time that the file is saved to the file system. Note that it is not necessarily the date that the file was originally saved. How can this be? Two things commonly affect the create date. If a user copies a file from one location to another, even though the two files are identical, each will have a different create date. The source file will show the date it was initially saved to that disk, while the new copy will have a create attribute that shows the time and date that it was first saved to the target drive.

The other way that create-attribute time stamps are modified is through a file system utility that allows a user to intentionally modify the attribute. There are several commercial and shareware applications that allow this. Therefore, by itself the create attribute doesn’t prove much of anything. It serves only as supplemental evidence to support other findings. Most applications that are used to generate files also embed creation metadata within the file. If a comparison of the two values shows a difference, there is sufficient cause for the investigator to look more deeply.

Access

The access attribute is the most volatile attribute of a file. Any time any user views, opens, copies, or backs up a file, this attribute is modified by the file system. Each time an executable is run, its access time is modified. Even the activity of antivirus scanning software has been known to alter the access time stamp. In fact, merely right-clicking on a file in Explorer and selecting Properties alters the access time stamp. There is no way for the investigator to accurately ascertain what action was invoked upon the file—only that one of them was. Many applications provide far more detail in their metadata concerning access information. For example, using the proper utilities, it is possible to identify the previous ten times that a document was accessed.

Modify

The modify time stamp is arguably the most valuable of the time/date attributes contained within a file. This information tells when the contents of the file were last altered. Any change to the file content sufficient to alter its hash value (which is virtually any change at all) is sufficient to reset this value. Actions that change the access and create attributes do not impact modify times. The act of moving or copying a file has no impact. These actions, however, will likely impact the attributes of the folder containing the files. For example, if a user copies NOVEL.DOC from C:\Documents to C:\User\Documents, the attributes of NOVEL.DOC will change as follows:

• SOURCE FILE ENTITY — C:\Documents\NOVEL.DOC—Create time remains the same, access time is reset, modify time remains the same.

• SOURCE FILE CONTAINER — C:\Documents—Create time remains the same, access time is reset, modify time remains the same.

• DESTINATION FILE ENTITY — C:\User\Documents\NOVEL.DOC—Create time is reset, access time is reset, modify time remains the same.

• DESTINATION FILE CONTAINER — C:\User\Documents—Create time remains the same, access time is reset, modify time is reset.

Entry Modified

The entry modified attribute (unique to NTFS) is modified each time any of the other three attributes is changed for any reason. It basically says that something in the metadata that comprises the MFT entry for the file has changes. There is no indication of which attribute changed. By itself, this tells the investigator little, if anything. However, it does suggest that further examination is in order.

The MAC time stamps can all be easily viewed in Windows Explorer or in one of the Linux File browsers. Figure 9.6 shows these attributes displayed in the file properties of a file stored on a Windows machine. (The problem with this approach is that merely viewing the file alters the accessed time stamp and is not acceptable procedure in the investigative process.) The entry modified attribute is not so easily viewed.

Figure 9.6 Windows Explorer is capable of displaying the Created, Accessed, and Modified file attributes.

Using MAC

One of the first things an investigator does when approaching a new inquiry is to ask “Who did what, and when did they do it?” The who part is usually the more difficult question to answer, although the when can usually be narrowed down to a relatively narrow time frame. Once a specific time has been identified, it might be possible to identify the users who had access to the data or to begin the search for who might have gained access from beyond the network. Many applications feature filtering functions that assist in this task. Figure 9.7 illustrates a simple filter (a function of Directory Snoop) to locate all document files that were modified on a specific date. Figure 9.8 shows the results of that filter.

Figure 9.7 A simple filter to search for files modified on a certain date

Figure 9.8 The results of the filter illustrated in Figure 9.7

Obviously, a typical investigation will involve many different file types and events. Commercial forensic software often features a timeline functionality that allows multiple queries to be processed simultaneously. Once a time frame has been identified, the investigator may want to identify all files created or modified during that window. Perhaps it is important to know that a specific application ran during that time period.

The Sleuth Kit, an open-source application running on the Linux platform, can generate timelines on virtually any file system. With The Sleuth Kit, the process consists of two parts. The fls utility collects the temporal data associated with each file on the system and collects it into a single file, called the body file. This is basically just a pipe-delimited ASCII text file that contains one line for each file, listing MAC data. (Pipe-delimited simply means that the file is a text file holding records, and each record is separated by the pipe character—the one that shares the backslash key over the right Enter key.) Once the body file has been built, the MACtime program can be used to build a timeline based on parameters selected by the investigator. The Sleuth Kit includes the ability to build the timelines as do all commercial forensic suites.

Keep in mind that the full timeline of a case is produced from more than simply file access or creation times. A full timeline includes data extracted from

• MAC data

• System logs

• Event logs

• E-mails

• Internet history

• File metadata

Analysis of these items is covered in other parts of this book. Too much reliance on file system metadata can be dangerous, since antiforensic utilities allow knowledgeable users to alter that data to read whatever they want it to.

Document Management Systems

Departmental or enterprise document management (DM) systems generate large amounts of metadata that gets stored in separate files maintained by the application. While the overall process of document management is beyond the scope of this book, it is important to understand that when such a system is in place, information that can be critical to an investigation or inquiry can be found in places other than the file system or from within the file.

Different developers approach document management in a variety of ways. Therefore, it is not possible to review every possible configuration or option; there are some generalities that are common between most products. Virtually all of them offer version control, check-in/check-out functionality, auditing, and history tracking. The actual documents are stored as files in an archive. The document archive may be called an archive, or it may be a cabinet, a repository, or a directory. It is not uncommon for an archive containing hundreds or even thousands of documents to exist as a single file managed by a database, such as Oracle or SQL Server.

The typical document management solution is client/server based. A common configuration is to have a database server handle the functions of security and indexing, and a dedicated storage device such as a Storage Area Network (SAN) or Network Attached Storage (NAS) head to warehouse the data. The files are not necessarily of the same type. A DM solution might host PDF files, TIFF files, document files, spreadsheets, and multimedia files—all in the same cabinet. The application can be programmed to automatically perform automatic document retention/destruction policies. For example, if a loan application is intended to be kept for seven years after the loan is rejected, the DM solution might automatically delete the file at seven years and one day. Any time there is potential for litigation, there should be a preservation order requested to immediately suspend such automatic destruction of documents and logs and to preserve all files that may be relevant to an investigation.

Embedded and Substantive Metadata

The majority of applications that create or edit user files also generate important metadata that is contained within the file structure. Some of this information is viewable within some functionality of the program, while some of it can only be examined with specialized applications. Some of the embedded metadata is generated by the application, while some of it can be added by the user. Figure 9.9 shows some of the viewable metadata of Microsoft Word at the time of writing. Much of the metadata defined as substantive falls under the category of embedded metadata. Substantive metadata includes information such as embedded fonts, line and paragraph spacing, indents, and so forth.

Figure 9.9 A compilation of four of the five Properties screens from a Microsoft Word document reveals the viewable metadata of this application. The fifth screen is of little or no relevance to the investigator.

An interesting thing to note in this image is that in the General Properties screen, the create date is reported as May 3, 2010, yet in the Statistics screen, it is shown as having been created on May 11, 2010. This is because the General Properties screen extracts its information from the OS metadata, while the Statistics screen represents embedded metadata. The difference is most likely the result of a fatal system error that locked up the author’s system on May 11. On reboot, the file was opened from the Microsoft document recovery option. Apparently, this altered the create date within the embedded metadata. This is a perfect example of how metadata cannot always be taken at face value, but requires intelligent analysis by the investigator.

Figure 9.10 is the final Properties screen as displayed by Microsoft. Virtually all of this information is data that can be entered by the user. These are considered custom metadata fields. For the purposes of illustration, the author has filled in some of the fields available.

Figure 9.10 Some applications allow the user to add personalized information in custom metadata fields.

In a situation where the target of the investigation was not technically astute and paid no attention to embedded metadata, much can be gleaned from an analysis of this data. If the document is being compared to a control document, a difference in revision numbers might suggest that the document was altered between the times that the two documents were altered. Also, the field in Statistics identifying the last user to save the document might be different from the document owner, if such information is saved in the Custom field, or from the author as saved in the Summary field.

A more technically astute suspect is very likely to alter this information. There are numerous shareware and freeware applications on the Internet, as well as commercial applications, whose sole reason for existence is to alter or remove metadata from documents. Therefore, if a user wants to hide this information, it isn’t difficult.

Another thing to consider as a result of this discussion is the nefarious use that the metadata files can be put to. In this particular document, the author has embedded the first two pages of this chapter into the Comments field in the summary tab (Figure 9.11). A drug dealer could store a list of suppliers, a pimp could have a complete listing of available talent, or hundreds of phone numbers could be hidden in a similar manner.

Figure 9.11 A relatively large amount of user data can be stored in the hidden metadata fields of a typical document.

Each application that creates embedded metadata can read its own. There is some compatibility between applications of a suite, but not a lot. For example, the embedded metadata of a PowerPoint file can indicate how many slides are in a presentation. Of what use is that to Word? Music files can host a large amount of data viewable in applications such as iTunes or Windows Media that identifies the song title, composer, artist, and dozens of other bits of information. Figure 9.12 shows just a fraction of the data available from the metadata of typical music files.

Figure 9.12 Most files created by user applications embed large amounts of custom metadata into the file package.

The majority of document metadata is easily viewable from within the application that created the file. Viewing the document properties exposes most of the metadata in Word, and there is a wide variety of metadata editors out there for image and music files.

Some critical metadata is hidden. A rather embarrassing incident happened to the British government in 2003, when a spokesperson released a dossier supposedly produced by their intelligence services. Dr. Glen Rangwala of Cambridge University was able to prove that the document was actually produced by a U.S. researcher by extracting the revision logs from the document (Thompson 2003). These logs are not visible through any conventional method but can be viewed by way of several third-party utilities. One utility called DocScrubber is a free utility from Java-Cool Software that can do this. Figure 9.13 shows the metadata uncovered from a document by this utility that reveals eight different revisions by three authors (only two of which appear in the screenshot).

Figure 9.13 Revision logs can reveal much about a document that is not visible from the application.

A sample of useful data that can be extracted from a documents metadata includes

• User name

• User initials

• Organization name (if configured on the system)

• Computer name

• Document storage location

• Names of previous authors

• Revision log (Word, Excel)

• Version log (Word)

• Template file name (Word, PowerPoint)

• Hidden text (Word, Excel)

• Globally Unique Identifiers (GUIDs)

Most applications can be configured to not save certain information, and there are also third-party utilities that can “cleanse” a document of this information. Therefore, it should not be expected that every document will expose each and all of these bits of data. For an example of examining a specific document, see the sidebar entitled Digging Deeper into the Document History.