Wide Web, tunneled over port 80 requests. Such requests cannot be filtered by a firewall in the traditional sense.

12.13 Intrusion detection and forensics

In the last few years the reality of network intrusion has led to several attempts to build systems which can detect break-ins, either while they are in progress or afterwards.

There are several ways in which we can gather evidence about intrusions. Evidence can be direct and indirect. Direct evidence might come from audits and log files, smoking guns, user observations, records of actions conducted by intruders, and so on. Checksums of important files can detect unauthorized changes, for instance. Indirect evidence can be obtained by looking at system activity and trying to infer unusual activity. Changes in the behavior of programs can signal changes in the patterns of usage of a system, perhaps flagging the exploit of a vulnerability in software.

Intrusion detection by process monitoring is a relatively new idea. The idea is to gather a profile of what is normal and compare it with software behavior over time. This idea is a little like the idea of an immune system which tolerates ‘self’ and reacts against ‘non-self’. Forrest et al. have pioneered system call profiling, inspired by vertebrate immune systems [119, 156] in order to detect hostile patterns of activity in special software processes. They build a database of short patterns of system call usage and then perform direct pattern search on subsequent data to detect anomalous patterns. The rationale for this approach is that intrusions are often caused by exploits of system calls which do not follow intended patterns. The beauty of this approach is its natural simplicity; its disadvantage is that it incurs a high overhead in resources to implement pattern searching in real-time; also the system needs to be taught what is normal in advance. Unfortunately ‘normal’ is a rather fickle concept [54], so in spite of its appealing simplicity, this is unlikely to be a complete, workable solution to the problem.

Another approach is to go to the network level and examine the totality of traffic arriving at a host. In order to detect an intrusion in progress, programs like Network Flight Recorder [102] (NFR) and Big Brother [236] (Bro) attempt to examine every packet on the network in order to look for tell-tale signatures of network break-in activity. This is an extremely resource-consuming task and it is beset with a number of problems. Few organizations have the resources to actually analyze the volumes of data they collect.

Network monitors look for packets containing data which might represent an attack, as they arrive. Network monitoring has its problems, however. One problem is that of fragmentation. Fragmentation is something which occurs to IP datagrams which pass between networks with different transmission rates. Larger packets can be broken up into smaller packets in order to optimize transmission. These fragments are reassembled at the final destination. This presents a problem for intrusion detection systems because the fragmented packets might not contain enough data to identify them as hostile. This would allow them to get past the detection system. An intruder might be able to generate packets which were fragmented in such as way as to confound the attempts at detection. Another

problem is that switches and routers limit the spread of traffic to specific cables. An intrusion detection system needs to see all packets in order to cover every attack. In spite of the difficulties, network intrusion detection is a hot research topic. A number of conferences on intrusion detection methods have sprung up to explore this problem in depth.

Network forensics is what one does after an intrusion. The idea is to examine logs and system audits in order to name the intruder and determine the damage. Network forensics is perhaps most important for the purpose of possible legal action against intruders. The cost of keeping the necessary logs and audits is very great and the work required after a break-in is far from trivial. This topic is beyond the scope of this book. See ref. [321] for an introduction, and the coroner’s toolkit (search the Internet for the nearest repository).

12.14 Compromised machines

Once an intrusion has been detected, one should not necessarily give away knowledge of the intrusion until all possible information about the intrusion has been collected. One should judge the risk of allowing the hack to continue: if the risk is acceptable, then important clues can be gathered by observing activity for a time. Keep a set of basic program tools on a read-only medium like a CD-ROM so that you are certain that you are not using Trojans. This should include a trusted shutdown command (halt).

1.Do not shut down the system, or pull the network plug until you have attempted to secure the volatile information in the system (process table, open port table).

2.Check that programs such as netstat, ps and halt are trusted, i.e. have not been replaced with modified programs. (If you are prepared, you will have an integrity check in advance.) If not, copy trusted versions from another system to /tmp, or use a diskette or CD-ROM containing trusted versions. Do not try to replace them while the system is running.

3.We want to avoid the activation of planted booby-traps, or logic bombs.

(a)Look for open ports with netstat – are there any open connections that can lead you to attackers? If you have a packet-based IDS, you might be able to see this information elsewhere.

(b)Look at all running processes and dump this information to a file for later examination. Do several dumps with different options.

(c)Hit the reset switch (if it exists) or pull the power plug or suspend the operating system: on Windows or Unix, a controlled shutdown using a trusted program is probably best, in order to ensure synchronization of caches with disks. On Windows 95, 98, ME and MacOS prior to version X, pulling the plug is good enough. Do not try a controlled shutdown unless you have a trusted copy of the halt program – you might set off a logic bomb planted by the cracker.

4.Look for files planted in user directories and packet sniffers. Script-kiddy software often leaves known filenames that can be searched for. Hackers will often try to divert attention from themselves by placing files in another user’s home directory – often several copies in case one is found.

5.The system should not be rebooted until necessary evidence has been secured. Indeed, the compromised machine should not be returned to normal service without a complete overhaul. Investigation should normally proceed by connecting the disk to a different, trusted computer. Note that some operating systems write data to disks when the OS starts, so disks should be isolated, write protected and copied before analysis is attempted. Special tools can be obtained for forensic work on IDE disks; SCSI disks can be protected by hardware jumpers.

A recovery policy should be in place as to what to do with forensic data. Will it be followed up by reporting to the police and then prosecution? The system should not be rebooted or the disk altered in any way.

Exercises

Self-test objectives

1.What elements should you have in a security and recovery plan?

2.Suggest some simple safeguards to protect inexperienced users from themselves.

3.What is meant by a file integrity check?

4.What is meant by a public-private key pair?

5.What is meant by a digital signature? Can such a signature be trusted?

6.Explain the significance of a trusted third party.

7.Give an example of an alternative to the use of trusted third parties.

8.Explain the assumptions that lie behind the security of the HTTPS Web protocol.

9.Is Kerberos a public-private key system?

10.Is Kerberos as secure as, say, SSL?

11.Describe how you would go about gauging security at a site.

12.What is meant by password sniffing?

13.How do one-time passwords work?

14.What is port scanning?

15.Explain the idea behind a virtual private network.

You see two attempts at protecting the directories for user ‘mark’. The order of the entries is slightly different. Do these ACLs yield the same protection? Are Mark’s private files properly protected in both cases?

7.Which of the following are software for file change detection?

(a)Tripwire or cfengine

(b)Snort

(c)Network Flight Recorder

(d)LIDS

(e)Trustix

8.Imagine that you were recommending a security strategy to a company. Which of the following priority lists would you recommend for the most cost-effective security?

(a)i. Security consultant service contract (outsourcing)

ii.Network Intrusion Detection Systems

iii.A firewall (network access control)

iv.Encrypted Virtual Private Networks

v.Strong authentication and access controls

vi.Smart cards for employees

(b)i. A firewall (network access control)

ii.Network Intrusion Detection Systems

iii.Strong authentication and access controls

iv.A security policy for prevention and response

v.Penetration testing

vi.Personnel security training

(c)i. A security policy for prevention and response

ii.File integrity checks on all machines

iii.Strong authentication and access controls

iv.Personnel security training

v.A firewall (network access control)

vi.Encrypted Virtual Private Networks

9.Try port scanning a part of your network. Be sure to inform the local system administrator of this in advance – it might be viewed as a hostile act.

10.Set up cfengine to perform a file integrity check on important system files on any hosts that you have privileged access to.

11.Are there any security risks associated with network printers? If so, what are they and how can they be removed?

12.Suggest ways of protecting against denial of service attacks from outside your company network.