State of the Practice of Intrusion Detection Technologies

Executive Summary

Attacks on the nation's computer infrastructures are a serious problem. Over the past 12 years, the growing number of computer security incidents on the Internet has reflected the growth of the Internet itself. Because most deployed computer systems are vulnerable to attack, intrusion detection (ID) is a rapidly developing field. Intrusion detection is an important technology business sector as well as an active area of research.

Vendors make many claims for their products in the commercial marketplace so separating hype from reality can be a major challenge. A goal of this report is to provide an unbiased assessment of publicly available ID technology. We hope this will help those who purchase and use ID technology to gain a realistic understanding of its capabilities and limitations. The report raises issues that we believe are important for ID system (IDS) developers to address as they formulate product strategies. The report also points out relevant issues for the research community as they formulate research directions and allocate funds.

Implementing intrusion detection systems on networks and hosts requires a broad understanding of computer security. The complexity of information technology infrastructures is increasing beyond any one person's ability to understand them, let alone administer them in a way that is operationally secure. Vendors are rapidly releasing new ID systems and aggressively competing for market share in an expanding market. Many products started out as point solutions. However, in response to consumers' inability to fully understand and use many ID systems, vendors are attempting to integrate approaches to solve a broader range of computer security problems. Evaluating ID systems is non-trivial and there is a lack of credible, comprehensive product evaluation information. Hiring and retaining personnel to competently administer security in general and intrusion detection in particular are increasing challenges. All of this rapid change makes it very difficult for an organization to implement an effective, long-term security strategy.

After reviewing the surveys cited in this report, one could conclude that ID technologies are becoming an accepted part of many organizations' information security tool suite. We are concerned that organizations are counting on these tools to solve a class of problems before they fully understand them. As a result, the solutions are likely to be inadequate or incorrect. Over-reliance on ID technologies can create a false sense of confidence about the degree to which tools are detecting intrusions against an organization's critical assets.

Both through our own experience and in discussion with technology experts and market analysts, we have observed that the current market condition of commercial ID tools and technologies exhibits a growing "bandwagon" effect. Each organization is comparing what they are doing with others in their peer group or market segment. If an organization views itself as taking security protection actions (such as deploying an IDS) that are equal to or slightly better than an organization that it considers its peer, that is good enough. At the decision-making level, there appears to be little or no regard for what ID systems can actually do. Nor is there an appreciation for the tasks that ID systems should not (or cannot) be relied upon to perform. Management's priority appears to be to ensure that they can demonstrate that they have exercised a standard of due care in the event of any legal action. We believe that the vendor community is marketing to this condition through the product claims they make.

It remains to be seen whether or not intrusion detection technology can live up to the promise of accurately identifying attacks. The current generation of commercial ID systems uses a limited set of techniques to detect attacks. Attackers are rapidly improving their abilities to penetrate networks successfully¾for example by developing ways to defeat ID systems themselves. Challenges to today's ID systems include

increases in the types of intruder goals, intruder abilities, tool sophistication, and diversity, as well as the use of more complex, subtle, and new attack scenarios
the use of encrypted messages to transport malicious information
the need to interoperate and correlate data across infrastructure environments with diverse technologies and policies
ever increasing network traffic
the lack of widely accepted ID terminology and conceptual structures
volatility in the ID marketplace which makes the purchase and maintenance of ID systems difficult
risks inherent in taking inappropriate automated response actions
attacks on the ID systems themselves
unacceptably high levels of false positives and false negatives, making it difficult to determine true positives
the lack of objective ID system evaluation and test information
the fact that most computing infrastructures are not designed to operate securely
limited network traffic visibility resulting from switched local area networks. Faster networks preclude effective real-time analysis of all traffic on large pipes.

ID systems can provide useful, reliable results in specific situations and configurations. These include monitoring an organization's firewall policy to ensure it is implemented correctly, monitoring unpatched machines for specific vulnerabilities, and monitoring specific network services.

The key deployment consideration is to focus the IDS sensing and analysis activities on the most critical subnets and hosts so that a trained analyst can interpret and act on the data these activities produce to safeguard the most important assets.

This report presents recommendations for ID sponsors, users, vendors, and researchers. For sponsors, we recommend

supporting ongoing, comprehensive testing of commercial ID systems and making test results publicly available
emphasizing research funding directed towards reducing false alarms

For users, we suggest

implementing a security architecture that reflects a defense-in-depth or layered approach to protecting an organization's assets, whether or not the organization chooses to deploy an IDS
developing clear, concise IDS requirements based on security policy and organizational needs
configuring the IDS to maximize performance. This includes selective deployment to monitor critical assets as well as signature tuning to prevent excessive false alarms.

We recommend that vendors

support initiatives to create open source signatures
move towards the distribution model used by the anti-virus community
spend more time and resources testing signatures and making results public
provide measures that represent the level of confidence a user should place in an ID system's ability to report an intrusion by type of signature or attack
integrate human analysis as part of event diagnosis
integrate available data sources more effectively to include information from different sensors and from different ID systems
expand options for capturing forensic evidence
increase efforts to detect malicious code (email attachments, Java, ActiveX)
increase interaction with the research community

We believe that the research community can benefit the ID field by

emphasizing the integration of diverse sources of available data to reduce false alarms
providing credible, defensible test data to support test and evaluation of ID systems
providing a taxonomy of vulnerabilities, i.e., a taxonomy that takes a victim rather than an intruder perspective
developing approaches for defending against sophisticated attacks such as denial of service, insertion, evasion, and distributed, coordinated attacks
developing approaches that integrate human analysis as part of event diagnosis
developing approaches that support better detection of malicious code
increasing interaction with the vendor community

This report does not emphasize current Department of Defense (DoD), Air Force (AF), and Defense Information Assurance Program initiatives in intrusion detection systems and technologies. Many of these efforts are specific to the DoD and involve proprietary products, systems, and documentation. In addition, we believe that the DoD and AF are well informed on the ID-related initiatives they are sponsoring. They are supported by other federally funded research and development centers (FFRDCs) (such as MITRE) in this area. Thanks to the Air Force Information Warfare Center (AFIWC), we have included a brief description of Government Off-the-Shelf (GOTS) ID efforts in Section 2.1.4. Our general approach was to analyze publicly available sources that could be of potential use to the DoD and to the general consumer, vendor, and research communities.

[Title Page] [Abstract] [Figures] [Acknowledgments] [Executive Summary] [Preface] [Section 1] [Section 2] [Section 3] [Section 4] [Section 5] [Appendix A] [Appendix B] [Appendix C] [Appendix D] [Appendix E] [Appendix F] [DTIC page] [PDF file]