There was something suspicious about the employee's actions. E-mail logs and network node analysis showed that every day at 5 p.m., he sent a photo to a Hotmail account. The photos, of the family dog, his car, his wife and children, looked innocent enough. But something was odd. For one thing, there was never any traffic back from the account.

The company's information security personnel watched this for several weeks and wondered what he was doing, until it occurred to them that these seemingly innocent photos could be a means of corporate espionage. Through the use of a technique called steganography, in which the digital ones and zeros of digital text or images can be buried inside the digital ones and zeros that make up the pixels of photographs, the employee could be secretly sending information. At the company's request, computer forensic consultants paid an after-hours visit to the employee's computer and found a copy of a steganographic program. The consultants also tested photographs on the employee's computer and found that information had indeed been buried in the photos. The employee was then confronted with the evidence in a private meeting with managers. He confessed to selling company secrets.

This case (based on a real incident with some details changed) illustrates how e-mail has become a potential security vulnerability. It is a virtual door that leads directly to the corporate network and indirectly to every desktop. It can be used by hackers to sneak in or, as in this example, by staff to sneak proprietary secrets out. It can also provide a portal for data destruction. Companies must be as adept at monitoring and controlling the traffic through this entryway as they are at monitoring physical traffic into and out of the corporate headquarters.

As with physical access controls, electronic controls can never achieve a zero-incident level. Thus, the security team must also know how to respond to a breach--how to follow the electronic trail to solve the case. That was essentially what law enforcement officials did when the I LOVE YOU virus hit businesses around the world. The e-mail trail ultimately led them to the originator in the Philippines.

This type of forensic tracing of e-mail is similar to traditional gumshoe detective work. By verifying each point through which an e-mail passed, the tracer works step-by-step back to the originating computer and, with luck, the perpetrator. To see how the process works, we'll walk through the basic steps an investigator follows when faced with an e-mail incursion.

Headers.

Most tracing of external e-mail starts at the receiving PC with an e-mail's Internet message-header information. A message header is text at the top of an e-mail that travels through the Internet. It contains the source of an e-mail in the "From" line, while in the "Received" lines, the header lists every point the e-mail passed through on its journey, along with the date and time. It's like having each post office that handles a letter print its identity, date, and time on the envelope.

The message header provides an audit trail of where an e-mail has been. Finding the individual who sent the e-mail is a matter of walking back up the audit trail point by point and gathering evidence that the message passed each point.

Let's consider a sample e-mail received by company ABCD, which shows four stopping points between sender and receiver in its header. Two of them, steps three and four, are within the company's e-mail system and will appear on the company's internal e-mail logs if they are logging that information.

If the second two points come from unfamiliar machine names outside the company's network, investigators can turn to some sleuthing tools such as Whois and Better-Whois, both commonly used in these types of investigations. These services search databases of registrars that record online users and their Internet Protocol (IP) addresses (numerical identifiers for computers on a network, the virtual equivalent of a street address). For example, running a Whois search on a domain name such as ABCD.com will identify the name and address of the domain name's holder, administrative and technical points of contact, and the domain name servers responsible for the domain. This gives investigators a place to start tracing the message.

If the address of the sending individual is not faked, finding the person behind the computer becomes a matter of determining who used the machine at the time the message was sent. For example, in one case where an individual sent a bomb threat to a company through a commercial e-mail account from a library computer, investigators traced the e-mail back to the library computer and were subsequently able to determine who had used the computer from the library's sign-on logs.

Faking it.

But forensic tracing is rarely this easy. Some e-mail systems strip the message header from the message before delivering it to the recipient or bury the message header within the e-mail program. In other cases, the "From" line in a message header is faked. Whois and Better-Whois can help trace the interim steps that the e-mail has taken, but they cannot determine the real address if phony information has been inserted, nor can they identify who has stolen another's account or used bogus address information, so any time a sender has faked an address, these tools are of limited use.

The "From" line can be faked in several ways. These include spoofing, remailing, relaying, stealing accounts, and creating bogus accounts.

Spoofing. Making an e-mail appear to come from someone (or someplace) other than the real sender (or location) is called "spoofing." The e-mailer uses a software tool that is readily available from the Internet to cut out his IP address and replace it with someone else's address. Fortunately, this trick does not create an impenetrable barrier for investigators, because the first machine to receive the "spoofed" message records the real IP address of the machine sending the message even though the faked ID is in the header. This provides the forensic examiner with a way to find the real machine and then begin to trace the individual who used it.

Remailing. Another way to throw investigators off the trail is to send e-mail to a computer that strips the sender's IP address and remails it with the remailing computer's IP address. The only way to find out who sent the mail is to get access to the logs of the computer that remailed the e-mail. But since remailer computers are designed to be anonymous, they usually don't log e-mail that has passed through them.

It is difficult to identify e-mailers who have used remailers unless they have made a mistake. One of those mistakes can be in the content they send. An analysis of the message or attachment, for example, can give clues to the identity of the sender. Information embedded by the software into the documents themselves can also give clues to the identity of the system and computer that the message came from.

Relaying. A third way that someone may hide the origin of an e-mail message is to have someone else's mail server send the message. A properly configured mail server will only process mail from within its system and won't relay mail from IP addresses originating from outside its network. But if the mail server is not correctly configured, it becomes vulnerable to misuse.

Spammers, for example, could create an e-mail message with a large number of recipients, then route the message through an unsuspecting company's mail server. The e-mailer uses it as a relay point, and the owner of the server may never be aware that the e-mailer has been there. The e-mailer then disappears before anyone gets suspicious. This is not only a theft of services, but potentially a denial of services as well, if the volume of e-mail sent through the server causes it to crash.

Stealing e-mail accounts. A fourth means of covering one's electronic tracks is to gain access to someone else's password and e-mail account. Some common ways in which access can be gained are shoulder-surfing (watching over someone's shoulder as they enter a password and ID), or "sniffing" a network (watching all the network traffic and intercepting user IDs and passwords).

Once a hacker has a legitimate ID and password, the whole network is compromised. When investigators get wise to the illegal activity and attempt to find the person behind it, they will be led to the innocent victim whose account has been hijacked. However, to determine who had hijacked the computer, investigators would need other ways to prove who the user was at the time of the crime. Public terminals may have a sign-in sheet or a surveillance camera, which can move the investigation forward.

Bogus freemail accounts. Another tactic used by criminals is to make sure that the trail will go cold when the investigator moves from the electronic to the real world. In this case, an e-mailer does not hide the e-mail's origin in terms of the computer it was sent from. However, getting to that computer in an investigation will reveal nothing about the true identity of the criminal because the person will have given a false identity and address when opening the account. It is difficult to catch someone who has done this because the e-mail company never knows who opened the false account. Pornographers often use this trick.

The log.

In most cases, forensic tracing of e-mail relies on computer logs. A computer log is a record of each e-mail message that passes through a computer in a network. Ideally, investigators prove that e-mail traveled through a machine by looking up the message ID on a log of e-mail transactions together with the date and time the address was recorded.

Unfortunately, this ideal scenario is not typical. Problems arise when logs are lacking. Legal limits and international jurisdictional issues can also create tough challenges for investigators tracing e-mail.

No logs. The biggest challenge facing an investigator is the Internet service provider (ISP) that does not log e-mail. Smaller ISPs don't turn on the computer logging functions of their computers either because they have inadequately trained staff or because they don't want the responsibility of turning over information about their clients. Some ISPs keep only partial data, such as log-ins or FTP (file transfer protocol) transfers into and out of the machine. This can make the investigator's job difficult because there is not enough information to take the next step.

Limits of the law. Even when ISPs maintain sufficient records, they vary in their willingness to help investigators. Some readily produce computer logs to help an investigation, while others refuse to give up logs without a court order or subpoena. (They are legitimately concerned about finding themselves in court for violating the privacy rights of users.)

For the private investigator without the backing of the law, getting a subpoena may be difficult or impossible. To overcome this impediment to progress, investigators can work with law enforcement when investigating a crime for their company or a client. If law enforcement officers contact the ISP and inform them that a certain user is being investigated, the ISP is obligated by law to preserve any information they would have normally logged or collected, giving investigators the time to seek the legal authority to seize the relevant information.

ISPs are not required to escalate their monitoring activities, however. Thus, if they were not keeping a log to begin with, they are under no obligation to start doing so.

International incidents. Tracing e-mail and computer intruders across international jurisdictions can be quite difficult. In many cases, one must have the backing of a legal attaché and the State Department as well as support from the local country's law enforcement agencies. If the trail leads to a computer in a country that is not willing to help, little can be done. Investigators might be able to point to a specific computer in the country, but there is no way to prove whether the computer was hacked or precisely who sent the e-mail.

In-house searches.

If the problem originates on the company's own computers, the search is easier, at least in the sense that the company has physical access to the equipment as well as the legal right to conduct a search of the contents (assuming that log-on banners and/or user agreements are in place where users consent to the monitoring of the system and agree that the system is for official use only). This assumes, of course, that the company has good procedures and policies in place. For example, it should ensure that mail servers are configured properly to log e-mail transactions and that those logs are backed up on a regular basis.

The lead case in this article gave one example of a type of problem that could originate on in-house computers. Another type of problem where the evidence often resides on corporate equipment is where an apparent company insider releases proprietary information about the company to a newsgroup that discusses the company's stock. If analysis of the newsgroup's header (similar to the e-mail header) indicates that the e-mail was sent from one of the company's PCs, investigators will try to identify a reasonable number of staff computers to examine, based on the employees who had access to that particular proprietary information.

Once those PCs have been found, the analysis team obtains exact copies (called image copies) of the computers' hard drives. Any analysis on a piece of media should always be conducted from an image copy to avoid tainting the original evidence. Then, the team conducts a comprehensive review of these records. They are looking for file fragments or portions of any e-mails that contain specific references to the offending message. For example, if the user was using the public e-mail service Hotmail, investigators will check on the image copy for the browser's Internet cache showing where the user has been online. It will contain copies of the e-mails created on or sent or received via Hotmail. If the user has emptied the cache, or otherwise deleted messages, investigators can usually use undelete utilities to recover this information.

Investigators can also conduct network node analysis, an examination of all computer logs that may help determine the path that an e-mail or a hacker took. For example, Web and FTP servers keep logs of every request made to the server, and automated tools are available to compare logs and piece together patterns of information or similarities.

Trends.

There are worrisome trends that suggest e-mail tracing will become more difficult in the future. For example, some new products that promise secure e-mailing can strip e-mail headers like a remailer, encrypt the message, and even destroy it after a period of time. Someone using that type of product would be essentially untraceable. Fortunately, such products are not yet in wide use.

In a perfect world, no message would escape the audit trail provided by the Internet e-mail header, but the network world is not perfect. Smart programmers are always looking for--and finding--ways to get around the audit trail, and investigators must play catch-up when tracing e-mail. Nevertheless, tracing provides clues to the next step that investigators should take, and it will likely remain an essential part of computer forensics.

Tim Poole is chief of the Veridian Digital Forensics Center (VDFC) in Falls Church, VA. James Hansen is a senior computer forensics examiner with Veritect, a Veridian company that provides network security services and trusted network environments in several commercial vertical markets.