How reliable are the answers from these SLE and ALE calculations? If we are going to make decisions based on these calculations, we need to know how reliable they are. I spent a long afternoon with a gentleman who was trying to convince me to invest a lot of money in an intrusion-detection framework. This thing would do everything but wax your car: it had sensor fusion, automated correlation of vulnerabilities with incoming attacks, and even factored in virus reports in a very cool graphics display. "Best of all," he says, "it has an expert system."

He continued talking and I nodded from time to time, but I was already gone. I couldn't help but remember phrases from my artificial intelligence (AI) classes. How about this one, "The reason expert systems don't live up to their promise is that the rules we are putting in them aren't very good. The knowledgeable engineer interviews the experts in the field, but what we are learning is that the experts aren't very expert." Here is another, "One of the biggest problems with AI is when the system doesn't know what it doesn't know. In that respect, AI systems are exactly like people."

When we calculate SLEs and ALEs, we need to be sensitive to what we don't know, to the places we fudge the numbers, to the cases where the models just don't fit. "No problem," you might be thinking. "I have no intention of calculating SLEs." Umm, maybe you do something similar, but you do it in your head without a process or documentation.

I work in an organization that monitors networks, for instance, although I guess that doesn't come as a surprise. I was listening to a new employee briefing and they were told very clearly that pornography was forbidden and that if caught, the responsible employees would probably be escorted out the door and fired. Let's jump into the mind of one of these young new employees. Maybe he is curious to see whether the organization can detect him if he misspells a sexually oriented word on a search engine, or uses oblique references. The answer is probably yes. But then again, he might think, "Hmmmm, but I already know they don't have a sense of humor, the SLE is just too high." Well, maybe he wouldn't use those exact words, but you get my drift.

Might I share one more example of uncertainty in answers with you? In mid-February 1999, I attended a working group for Presidential Decision Directive 63 (PDD 63). The goal was to get the 50 or so top researchers (and me) to consider four problem areas necessary for allocating approximately half a billion dollars in research money for intrusion detection and information assurance. One of the tracks was called anomalous behavior, which is Washington D.C. speak for the trusted insider problem. So, we all worked away and then presented our results. The anomalous group presented a finding that research had been funded 100 times more for detecting outsiders than insiders. Someone asked, "What study did you find that ratio in, and what was your source?" The answer from our distinguished scientists was "We made it up, but it's close."

Risk Management Is Dollar Driven

If you approach management and say you need $10,000 for an intrusion-detection system, they might want a bit more information. It is a good sign if they ask how much time it will take to run such a system; it shows they are listening and thinking clearly. A good manager knows the hardware and software costs are the tip of the iceberg and wants to get a handle on the whole picture. Managers want to understand how it fits into the business model. Risk management (and that includes intrusion detection) is dollar driven.

Whenever we are faced with a risk that is unsavory to us, we begin to wonder what can be done to reduce or mitigate the risk. As we pick our countermeasures, we should try to calculate what they would cost on a yearly basis. When you make a proposal to management, people really like it if you can give the cost breakdown and even an option or two. Remember those SLEs and