Statistical-Based Intrusion Detection

Software Technology Roadmap

What's New

Background & Overview

Technology Descriptions

	Defining Software Technology
	Technology Categories
	Template for Technology Descriptions

Taxonomies

Glossary & Indexes

Feedback & Participation

Software Engineering Information Repository (SEIR)

Statistical-Based Intrusion Detection

Note

We recommend Intrusion Detection as prerequisite reading for this technology description.

Intrusion detection systems (IDS) automate the detection of security violations through computer processing of system audit information. One IDS approach, Rule-Based Intrusion Detection (RBID) , seeks to identify intrusion attempts by matching audit data with known patterns of intrusive behavior. RBID systems rely on codified rules of known intrusions to detect intrusive behavior. Intrusion attempts not represented in an RBID rule base will go undetected by these systems. To help overcome this limitation, statistical methods have been employed to identify audit data that may potentially indicate intrusive or abusive behavior. Known as statistical-based intrusion detection (SBID) systems, these systems analyze audit trail data by comparing them to typical or predicted profiles in an effort to find pending or completed computer security violations. This emerging technology seeks to increase the availability of computer systems by automating the detection and elimination of intrusions.

Technical Detail

SBID systems seek to identify abusive behavior by noting and analyzing audit data that deviates from a predicted norm. SBID is based on the premise that intrusions can be detected by inspecting a system's audit trail data for unusual activity, and that an intruder's behavior will be noticeably different than that of a legitimate user. Before unusual activity can be detected, SBID systems require a characterization of user or system activity that is considered "normal." These characterizations, called profiles, are typically represented by sequences of events that may be found in the system's audit data. Any sequence of system events deviating from the expected profile by a statistically significant amount is flagged as an intrusion attempt [Sundaram 96]. The main advantage of SBID systems is that intrusions can be detected without a priori information about the security flaws of a system [Kemmerer 94].

SBID systems typically employ statistical anomaly and rule-based misuse models [Mukherjee 94]. System profiles, user profiles, or both may be used to define expected behavior. User profiles, if used, are specific to each user and are dynamically maintained. As a user's behavior changes over time, so too will his user profile. No such profiles are used in RBID systems. As is the case with RBID systems, known intrusion scenarios can be codified into the rule base of SBID systems.

Interesting variations on this theme include the following:

Predictive pattern generation, which uses a rule base of user profiles defined as statistically-weighted event sequences [Teng 90]. This method of intrusion detection attempts to predict future events based on events that have already occurred. Advantages of this approach include its ability to detect misuse as well as intrusions and its ability to detect and respond quickly to anomalous behavior.
Connectionist approaches in which neural networks are used to create and maintain behavior profiles [Lunt 93]. Advantages of neural approaches include their ability to cope with noisy data and their ability to adapt to new user communities. Unfortunately, trial and error is required to train the net, and it is possible for an intruder to train the net during its learning phase to ignore intrusion attempts [Sundaram 96].

Usage Considerations

An advantage of SBID systems is that they are able to adaptively learn the behavior of the users they monitor and are thus potentially more sensitive to intrusion attempts than are humans [Sundaram 96, Lunt 93]. However, SBID systems require the creation and maintenance of user/system profiles. These profiles are sensitive to hardware and software modifications, and will need to be updated whenever the system or network they used to protect is modified. Additional work is required to determine how statistical user/system profiles should be created and maintained [Lunt 93].

Maturity

Statistical intrusion detection algorithms have been in existence since at least 1988. Several prototype systems have been developed, including Haystack [Smaha 88], IDES [Lunt 93], and MIDAS [Mukherjee 94]. MIDAS is a deployed real-time SBID that provides security protection for the National Computer Center's networked mainframe computer. IDES, which is deployed at both SRI and FBI locations, is an IDS that combines SBID with RBID to detect a wider range of intrusion attempts. Another deployed security system containing aspects of SBID technology is AT&T Bell Lab's Dragons system which protects their Internet gateway;¹ the Dragons system has succeeded in detecting intrusion attempts ranging from attempted "guest" logins to forged NFS packets [Mukherjee 94].

Costs and Limitations

In addition to the costs associated with creating audit trails and maintaining user profiles, there are several risks and limitations associated with SBID technology:

Because user profiles are updated periodically, it is possible for an insider to slowly modify his behavior over time until a new behavior pattern has been established within which an attack can be safely mounted [Lunt 93].
Determining an appropriate threshold for "statistically significant deviations" can be difficult. If the threshold is set too low, anomalous activities that are not intrusive are flagged as intrusive (false positive). If the threshold is set too high, anomalous activities that are intrusive are not flagged as intrusive (false negative).
Defining user profiles may be difficult, especially for those users with erratic work schedules/habits.

Like RBID systems, SBID systems will negatively affect throughput because of to the need to collect and analyze audit data. However, in contrast with RBID systems, SBID systems do not always lag behind the intruders. Detection of anomalous behavior, whether or not it is codified as a known intrusion attempt, may be sufficient grounds for an SBID system to detect an intruder.

Use of this technology requires personnel who are experienced in statistics and intrusion detection techniques and who have in-depth knowledge of audit collection mechanisms.

Dependencies

Expert systems are an enabler for this technology.

Alternatives

Other approaches to intrusion detection include model-based or rule-based approaches (see Rule-Based Intrusion Detection), and approaches based on genetic algorithms. Manual examination of recorded audit data and online monitoring of access activity by knowledgeable personnel are the only other known alternatives.

Complementary Technologies

Rule-Based Intrusion Detection systems can be used in conjunction with statistical-based intrusion detection systems to catch a wider variety of intrusion attempts, and user authentication systems can be used to help verify user identify.

Index Categories

This technology is classified under the following categories. Select a category for a list of related topics.

Name of technology

Statistical-Based Intrusion Detection

Application category

System Security (AP.2.4.3)

Quality measures category

Security (QM.2.1.5)

Computing reviews category

Operating Systems Security and Protection (D.4.6)
Computer-Communication Networks Security and Protection (C.2.0)
Security and Protection (K.6.5)

References and Information Sources

[Bell 76]	Bell, D. E. & LaPadula, L. J. Secure Computer System: Unified Exposition and Multics Interpretation Rev. 1 (MTR-2997). Bedford, MA: MITRE Corporation, 1976.
[Kemmerer 94]	Kemmerer, Richard A. "Computer Security," 1153-1164. Encyclopedia of Software Engineering. New York, NY: John Wiley and Sons, 1994.
[Lunt 93]	Lunt, Teresa F. "A Survey of Intrusion Detection Techniques." Computers and Security 12, 4 (June 1993): 405-418.
[Mukherjee 94]	Mukherjee, Biswanath, L.; Heberlein, Todd; & Levitt, Karl N. "Network Intrusion Detection." IEEE Network 8, 3 (May/June 1994): 26-41.
[Smaha 88]	Smaha, Stephen E. "Haystack: An Intrusion Detection System," 37-44. Proceedings of the Fourth Aerospace Computer Security Applications Conference. Orlando, Florida, December 12-16, 1988. Washington, DC: IEEE Computer Society Press, 1989.
[Spafford 88]	Spafford, Eugene H. The Internet Worm Program: An Analysis (CSD-TR-823). West Lafayette, IN: Purdue University, 1988.
[Sundaram 96]	Sundaram, Aurobindo. An Introduction to Intrusion Detection [online]. Available WWW <URL: http://www.acm.org/crossroads/xrds2-4/xrds2-4.html> (1996).
[Teng 90]	Teng, Henry S.; Chen, Kaihu; & Lu, Stephen C. "Security Audit Trail Analysis Using Inductively Generated Predictive Rules," 24-29. Sixth Conference on Artificial Intelligence Applications. Santa Barbara, CA, May 5-9, 1990. Los Alamitos, CA: IEEE Computer Society Press, 1990.

Copyright 2007 by Carnegie Mellon University
Terms of Use
URL: http://www.sei.cmu.edu/str/descriptions/sbid_body.html
Last Modified: 11 January 2007

Status