State of the Practice of Intrusion Detection Technologies
2 What Is the Current State of Intrusion Detection Technologies?This section covers a range of topics dealing with current ID technology and practice to illustrate where ID systems stand today. We start with a review of technology, looking at currently used tools in the research, commercial, and public domains. We then look at market conditions, summarizing several papers and surveys that describe the current market and where it is headed. We conclude by discussing some experiences with representative ID products that indicate why current ID systems are not the only solution to fix all security problems.
2.1 Survey of ID TechnologyID technology is immature and dynamic. Like the early auto manufacturers, new vendors appear, only to be absorbed by other vendors. The same is true with ID products (both commercial and research). Because of the rapid changes in the field, information such as lists, surveys, or reviews are quickly outdated. For example, a report by Staniford-Chen provides a summary of 42 ID-related products, but much has changed since its date of publication (winter 1997-1998).1 Web-based lists (such as the one found in the SANS/NSA ID tools inventory) are easier to keep updated [B4].
As part of this effort, we conducted a wide-ranging review of ID literature. This review focuses mostly on materials accessible via the Web. Appendix B contains references to the articles reviewed in this section while Appendix D contains selected reviews.
Intrusion detection has been an active field of research for about two decades. This is exemplified by an influential paper, published in 1980, "Computer Security Threat Monitoring and Surveillance" by James Anderson [B34]. It was followed some years later (1987) by the seminal paper "An Intrusion Detection Model" by Dorothy Denning [S7]. Denning's paper provides a methodological framework that inspired many researchers and, in more recent times, laid the groundwork for commercial products.
There are six main topic areas covered by the survey: ID surveys, taxonomies, testing and evaluation, research, commercial tools, and ID directions (all are further elaborated in Appendix D). The papers that are more relevant to this review are discussed more fully, while other papers are simply cited because they are useful resource materials. In all cases, the reviews are brief and are provided so that the reader can selectively target relevant papers. Each review follows the same structured table-based format.
This section presents some research and commercial products that are examples of available ID technology, as well as a few products available in the public domain.
2.1.1 Examples of Research ProductsID research performed in the early 1990s produced a number of new tools [S6]. However, many were developed by students to explore concepts, and after they moved on, the tools were not maintained. Nevertheless, these tools influenced the direction of subsequent research efforts and also of commercial ventures. Early efforts often focused on host-based solutions, but because of the explosive growth of networking, later efforts concentrated on network-based systems.
The tools reviewed here reflect a core of active research that has evolved from earlier efforts. The first two, EMERALD and NetSTAT, have matured a great deal and compliment each other's approaches. The third research tool discussed is Bro. It is unique in addressing the issue of network penetration through attempts to overload or confuse the ID system.
2.1.1.1 EMERALDEMERALD (Event Monitoring Enabling Responses to Anomalous Live Disturbances) is the most recent research tool developed by SRI International. This line of tools has explored issues in intrusion detection associated with both deviations from normal user behavior (anomalies), and known intrusion patterns (signatures). SRI's pioneering work in intrusion detection began in 1983 when a multivariate statistical algorithm was developed to discriminate between different user behaviors [R1-d].
Somewhat later, the use of a signature analysis subsystem, based on the P-BEST [R33] expert system was investigated to support detection of suspicious activities. These research efforts were incorporated into SRI's early intrusion detection system, IDES [R1-b], a system that monitors activity on multiple hosts in real time.
Based on experiences with IDES, a re-architected, production-oriented tool, NIDES [R1-c], was developed between 1992 and 1994. Like IDES, this tool is host-based, and uses the P-BEST production rule system.
However, it went further by adding a component called RESOLVER that fuses the results from the statistical and signature analysis components. The user interface in NIDES was also a significant improvement over the IDES user interface.
EMERALD builds on the earlier IDES/NIDES experiences but this time focuses on support for networks rather than for a collection of hosts. A major goal of EMERALD is to address issues associated with large, loosely coupled enterprise networks. Such environments are more difficult to monitor and analyze due to the distributed nature of the incoming information. EMERALD structures users into a federation of independently administered domains. Each domain provides a collection of network services, such as http or ftp, that may have different trust relationships with each other, and across which different security policies may apply. In this context, one centralized repository is likely to result in significant performance degradation, as is the centralized analysis of all the data. These issues motivated the work on EMERALD and the demonstration of "divide and conquer" techniques that it investigated.
The hierarchical approach provides three levels of analysis performed by a three-tiered system of monitors: service monitors, domain monitors, and enterprise monitors. These monitors have the same basic architecture: a set of profiler engines (for anomaly detection), signature engines (for signature analysis), and a resolver component that integrates the results generated from the engines.
Each module also contains a resource object that provides a configurable library of information to customize the module's components to the target application. This object can be re-used in multiple monitors within an EMERALD application. At the lowest level, service monitors support intrusion detection for individual components and network services within one domain, probing for or reading data (activity logs, events, etc.), and performing local signature and statistical analyses.
Domain monitors integrate information from the service monitors to provide a domain-wide view of intrusions, while the enterprise monitors perform inter-domain analysis to assess threats from a global perspective. Of interest at the enterprise level are such threats as worm-like attacks and inter-domain attacks on network services. Subscription-based communication channels allow different service monitors to communicate with each other, either by having information directed from one monitor to another ("pushed") or requested by one monitor from another ("pulled").
Prior work with NIDES demonstrated that statistical profiling techniques could be effective with either users or applications as targets. The monitoring of applications (e.g., anonymous FTP), was particularly effective since fewer application profiles were required. EMERALD generalizes the profiling technique by abstracting the notion of a profile, separating profile management from profile analysis.
With respect to signature analysis, service-layer signature engines monitor domain components to determine if abnormal activity is occurring through known exploit scripts. Signature engines in higher-level monitors distill this information to assess if a broader attack is occurring. In addition to integrating the results from the statistical and signature engines, the resolver component provides other functions. These include providing a subscription service that allows a third party tool to be integrated into the EMERALD environment, acting on reports generated by the statistical and signature engines, providing an interface to the monitor administrator, and initiating attack countermeasures, such as terminating processes.
EMERALD is a work in progress. It provides an example of the direction that future intrusion detection systems may take. As intruders become more sophisticated in their attacks, they will be increasingly likely to disperse the evidence of their work across networks, making it difficult to sense when a distributed/coordinated attack is occurring. In such situations, the ability to collect, assimilate, correlate, and analyze information emanating from diverse sources in real time becomes essential. The flexibility of EMERALD's scalable architecture, its ability to abstract functionality, its openness to the addition of external tools, and the prior experience (e.g., with NIDES) that is reflected in its functional components, makes EMERALD a forerunner of future ID tools. However, managing and maintaining the information base and building the system's infrastructure may require significant effort.
2.1.1.2 NetStatNetSTAT is the latest in a line of "STAT" research tools produced by the University of California at Santa Barbara. The STAT activity, started in the early 1990s, explores the use of state-transition analysis in support of real-time intrusion detection [R4]. The approach is based on the premise that certain sequences of actions reflect unauthorized activity and indicate an intruder moving the system from an initial authorized state to a compromised state.
Most host-based intrusion detection systems in the anomaly category analyze evidence for intrusion in the computer's audit trail. However, in the STAT approach, the audit trail information is transformed through an "audit trail analyzer" that filters and abstracts the information gathered at the audit trail level. These abstractions, which are more suitable for analysis, portability, and human understanding, are called signatures and are central to the STAT approach. Signature actions move the system through the sequence of states, each state driving the system closer to a compromised configuration. Intrusion sequences are defined by state transitions that are captured in production system rule-sets.
The initial implementation of the method was a host-based, UNIX-based system called USTAT [R4]. USTAT was composed of
- a preprocessor
- a knowledgebase (that included a fact base and rule base)
- an inference engine
- a decision engine
The preprocessor filters and manipulates the data into a form that is audit-file independent. The rule base component of the knowledgebase stores the state-transition rules that indicate the predefined intrusion sequences, while the fact base stores the dynamically changing state of the system with respect to possible ongoing intrusions.
Given new information generated by the preprocessor together with the current system state as defined in the fact base, the inference engine identifies any significant state changes and updates the fact base. An update function then revises the fact base to reflect these changes. The inference engine also notifies a decision engine of possible security violations. The decision engine in turn either notifies the site security officer of the event or initiates action on its own. One advantage of the state-driven approach is that an attack may be recognized and acted on prior to reaching the compromised state.
This state-based approach uses an inference engine table to track each possible intrusion, and allows USTAT to identify a coordinated attack emanating from multiple sources. It can do this since attack sequences are defined, not by who is perpetrating the attack but by states of the system. Thus, if two attackers are relying on the same composite state of the system, each of their subsequent actions can be followed through a fork in the previous state transition sequence. This forking is implemented by duplicating rows in the inference engine table, each row representing different attack sequences.
NSTAT [R3] was the natural successor to USTAT. NSTAT focused on supporting a network of hosts in which, for example, files are shared. Thus actions on one host, such as mounting directories, can influence other machines on the network. Having one centralized detection system results in less performance impact on the local hosts and also allows for more informed intrusion analysis when a multi-host attack is being perpetrated. With NSTAT, the local hosts convert audit data into NSTAT format and merge the data steams into one.
The most recent of the tools, NetSTAT [R30], is currently under development and diverges from the prior host-based systems by addressing network intrusion. NetSTAT is composed of a set of probes that are responsible for detecting and evaluating intrusions in the sub-networks to which the probes are attached. Each probe is supported by a remotely configurable data filter, an inference engine, and a decision engine. These probes can act autonomously. However, different parts of the network may detect components of an intrusion (because of the differing locations and filters). If an intrusion component is detected, then an event can be forwarded to other interested probes that subscribe to that event in order to get a more complete understanding of the intrusion. Thus, intrusions that involve separate subnetworks can be identified.
The probes are supported by an analyzer, a stand-alone tool that supports the generation and management of probes. The analyzer is composed of a network fact base, a database of state-based intrusion scenarios, an analysis engine, and a configuration builder.
It determines which events should be monitored for, where they should be monitored, what network topology information is required, and what network state information is required to support the intrusion analysis. To perform these actions, the analysis engine uses information in the network fact base together with the scenario database to define attacks to which the network may be vulnerable. This information is passed to the configuration builder that in turn generates the probe configurations. These probe files consist of a filter, state-transition information, and the decision tables that allow the probe to execute.
2.1.1.3 Bro
Bro is a research tool being developed by the Lawrence Livermore National Laboratory. It is being built, in part, to explore issues related to the robustness of intrusion detection systems, i.e., assessing what characteristics make an ID system capable of resisting attacks against itself. The design goals for Bro
[R31] include
Bro has a three-level hierarchy of functions. At the lowest level, Bro uses libpcap, a utility to extract packets from the network. This decouples the main intrusion detection functionality of Bro from the networking details. This also allows a significant fraction of the packets entering the network to be rejected at a low level. Thus libpcap will capture all packets associated with the application protocols (e.g., finger, ftp, telnet) of which Bro is aware.
The next layer, the event layer, performs integrity checks on packet headers. If the header is ill-formed, an event identifying the problem is generated, and the header is discarded. A check is then performed to determine if the full contents of the packet should be recorded (usually if the full packet was analyzed), if only the packet's header information should be recorded (usually if only the TCP flags were analyzed), or if nothing should be recorded (if no processing was done).
Events are generated from this process and placed on a queue to be interrogated by the policy script interpreter which resides in the third layer. The policy script interpreter is written in a customized Bro language that uses strong typing to provide explicit support for packet header content such as port and domain and other constructs to support networking concepts. The interpreter binds event values to the code for the event handler and then interprets the code. Executing the code may result in generating further events, logging real-time notifications, or recording data. To add new capability to Bro, one needs to identify the events associated with the protocols of the application, and write corresponding event handlers to extend the functionality of the policy script interpreter. The developer claims that this decoupling of events from their handlers improves Bro's extensibility.
At present Bro monitors four applications: finger, ftp, portmapper, and telnet. Adding new applications to Bro is, according the developer, "quite straightforward, a matter of deriving a C++ class to analyze each connection's traffic, and devising a set of events corresponding to the significant elements of the application"
[R31]. Bro runs under several UNIX variants and is used as part of the lab's security system. As of 1998, Bro's operation had resulted in the filing of 85 CIAC and CERT/CC incident reports. Bro experiences no packet loss on a FDDI network carrying 25mbs traffic with analysis loads of peaking at about 200 packets/second.
The commercial products described here represent only a small subset of those now on the market
(see the references [S12],
[B4],
[B3],
for product lists), and the reviews of these products do
not imply endorsement. Recent comparative evaluations can be found in references in
Appendix B
[S20],
[S21],
[S37]. The products discussed below represent a cross-section of approaches being employed in intrusion detection. In this context, we did not necessarily select the most dominant or fully functional products, but tried to pick those that reflected a cross-section of abilities spanning such dimensions as host-based vs. network-based and anomaly-based vs. signature-based. Also, there is a high degree of quality and detail in the product's literature. Currently most commercial systems are network-based. However, many commercial vendors are aggressively developing products that integrate host- and network-based approaches.
Unlike the research product literature, commercial product literature is generally weighted towards marketing. This can make it difficult to identify the functionality that the products actually support. For example, it requires close reading to identify a product's technical basis (is it host-based or network-based?). Virtually no commercial literature covers what are perhaps the most important topics to the prospective buyer: the frequency of false positives (generating unnecessary alerts) and the frequency of false negatives (not identifying intrusions when they occur). While work is progressing on developing standardized test data
[B29], accurate assessments of commercial tools are lacking.
The Computer Misuse Detection System (CMDSTM) was originally developed by Science Applications International Corporation
[C13], and is now maintained and marketed by ODS Networks Inc.
[C11]. It is a host-based system that supervises a (potentially) tiered network of machines. It supports the anomaly (statistical) and misuse (signature) categories of detection and can also generate analysis reports depicting trends.
In the anomaly category, CMDS uses statistical analysis to identify patterns of behavior that deviate from normal user practice. The statistics are derived from such categories as
Profiles are automatically generated during operation, broken down by hour. These profiles are examined for questionable behavior in each of the three categories (network, execution, and browsing). Deviations from expected behavior (over a period of one day) are computed and warnings are generated if deviations are above a threshold value.
Signature recognition is supported by the CLIPS expert system
[B136]. Facts, derived from event numbers, object names etc., are used to instantiate the CLIPS signature-based rules and to identify possible illicit activity. CMDS defines UNIX attack signatures associated with, for example, failed superuser attempts, login failures, vacation activity, and attempted critical file modification. It has an equivalent set of defined signatures for the NT operating system.
The tool allows many types of report categories. Based on the above analyses, reports by user, by machine, by date, and by hour can be generated. Within these categories, various graphical displays allow visual trend analysis to be performed.
NetProwlerTM
[C17], currently being released, comes bundled with Intruder Alert from Axent Corporation. The Intruder Alert component supports host-based intrusion detection while the NetProwler component (previously called ID-Track from Internet Tools, Inc.) supports network-based detection.
NetProwler incorporates what Axent calls the "Stateful Dynamic Signature Inspection" virtual processor. This provides a means to integrate small chunks of information being sniffed on the network into more complex events, to test events against predefined signatures in real-time, and to install new signatures while keeping the system running. NetProwler provides signatures for a wide variety of operating system and application attacks, and allows users to build customized signature profiles using a signature definition wizard. Examples of attack signatures that NetProwler supports include denial of service, unauthorized access, vulnerability probes, suspicious or malicious activity, and activity that is counter to company policies. The tool incorporates a "Profiler" that supports installation and configuration by probing the network's hosts and their applications to determine what attack signatures should be installed.
Using the attack signature definition (ASD) user interface, together with the attack wizard, users can define new attack signatures. In this way, users can characterize attacks that are composed of single events, repeated events, or series of events. An attack signature has four elements: a search primitive (a string pattern); a value primitive (a value or range of values); a reserved keyword (a protocol name such as IP); and the operating system or application associated with the attack.
NetProwler also supports automated response capabilities. This includes session logging, session termination, posting events on the event console, and alerting personnel through email, paging, and other means.
NetRangerTM from Cisco Systems
[C20] is a network-based ID system. It operates in real time and is scalable to the enterprise level. A NetRanger system is composed of Sensors and one or more Directors that are connected by a "Post-Office" communications system. Each of the Sensors is deployed on a Cisco hardware platform, but the Director is software-based.
Sensors are placed at strategic points on the network and interrogate passing network traffic. Sensors can analyze both the header and content of each packet, and can associate packets that have characteristics in common. Each Sensor can analyze single packets for attacks or can maintain state, allowing for the detection of multi-packet attacks. Sensors use a rule-based expert system to interrogate the packet information to determine the type of attack¾be it simple or complex.
Three categories of attack are recognized: named attacks (i.e., attacks with a specific name); general attacks (i.e., named attacks that have spawned many varieties); and extraordinary attacks (i.e., attacks with highly complex signatures). In addition to providing many standard attack signatures, NetRanger provides the ability for the user to define customized signatures. In response to an attack, a Sensor has several options that include generating an alarm, logging the alarm event, killing the session, and denying further network access.
The Director provides centralized management support for the NetRanger system. In this capacity, it allows remote installation of new signatures into the Sensors, and collects and analyzes security data. The status of the Sensors can be monitored through a color coding. Entities in the system (machines, applications, alarms, etc.) have states shown as text or icons, and each state is represented by a different color. Normal states are shown in green, marginal states are shown in yellow, while critical states are shown in red. The Sensors are managed through the Director (using a Java-based tool called nrConfigure) in order to make changes to functional areas such as communications, data management, intrusion detection, and collection of sensor data.
For performance reasons, the Director does not directly support a reporting capability. This is done through third-party database support. The Director pushes out logs to a staging area from which the logging data are loaded into the database. The database can then be queried for information such as a request to plot the number of port sweeps for each day that occurred during the last week. Director notifies personnel of events via email. User-defined notifications can also be implemented.
Until recently, Centrax Corporation sold a product called EntraxTM. However, in March 1999, Centrax was bought by Cybersafe Corporation. Cybersafe Corp. made some significant technical changes to Entrax and renamed it CentraxTM
[C12].3
Entrax originally focused on host-based detection. Reasons
included the ability of host-based systems to detect security holes in individual machines in the network, the inability of network-based ID systems to examine encrypted packets, and the limited ability of network-based ID systems to monitoring insider misuse. However, the current version of Centrax does include both host-based and network-based monitoring, a move that may have been influenced by the current popularity of the latter approach. Some reviews of Entrax/Centrax product line are included in the references in
Appendix B
[S24],
[S25].
Centrax provides two main components; a Command Console and a Target Agent. These are analogous to the Directors and Sensors in the NetRanger system. However, the Target Agent can be one of two types: one to collect host-based information, the other to collect network-based information. These Target Agents reside on the machines which they are monitoring (e.g. individual PCs, file or print servers) and relay the information back to the Command Console for processing.
For performance, the Network Target Agent is a stand-alone machine. The host-based agents support over 170 signatures (such as for viruses and Trojan horses, object browsing, and password changing), while the network-based agents support 40 signatures.
These host-based agents can detect and react to threats locally in real time. Each threat can have its own response pattern, such as terminating the connection to the offending machine.
The Command Console performs a variety of functions through its Manager and Editing components. The Target Manager downloads auditing and collection policies to the Target Agents, the Assessment Manager probes host machines for security vulnerabilities, and the Alert Manager displays information on detected threats and can respond to these threats by, for example, logging out a user or shutting down the computer. A set of editors provides the capability to organize policies in areas such as signatures (defining under what conditions to generate alerts), audit data collection, and customized reports. The Target Agents generate raw audit data which is archived, and after appropriate reduction, sent to a database. The database can then generate reports.
The Command Console runs on Windows NT, while the host based Target Agents run on either Windows NT or Solaris-based systems. The Network Target Agent runs under Windows NT.
RealSecure
[C15]
[S37] from Internet Security Systems is another real time IDS. It uses a three level architecture consisting of a network-based recognition engine, a host-based recognition engine, and an administrator's module. The network recognition engine runs on dedicated workstations to provide network intrusion detection and response. Each network recognition engine watches the packet traffic traveling over a specific network segment for attack signatures¾telltale evidence that an attempted intrusion is taking place. When a network recognition engine detects unauthorized activity, it can respond by terminating the connection, sending email or pager alerts, recording the session, reconfiguring selected firewalls, or taking other user-definable actions. In addition, a network recognition engine passes an alarm to the administrator's module or a third-party management console for administrative follow-up and review.
The host-based recognition engine is a host-resident complement to the network recognition engine. It analyzes host logs to recognize attacks, determines whether the attack was successful or not, and provides other forensic information not available in a real-time environment. Each host engine is installed on a workstation or host, and thoroughly examines that system's logs for tell-tale patterns of network misuse and breaches of security. The host engine reacts to prevent further incursions by terminating user processes and suspending user accounts. It can send alarms, log events, send traps, send e-mails, and execute user-defined actions.
All recognition engines report to and are configured by the administrative module, a management console that monitors the status of any combination of UNIX and Windows NT recognition engines. The result is comprehensive protection, easily configured and administered from a single location. The administrative module ships with either recognition engine and is also available as a plug-in module for a variety of network and systems management environments.
There are a number of freely available or public domain tools that can be used for intrusion detection. We reviewed two of these, Shadow and Network Flight RecorderTM,4 which are supported by a joint effort of the Naval Surface Warfare Center, Network Flight Recorder, Inc., the National Security Agency, and the SANS Institute
[R39]. These systems are unlikely to have the same level of support as commercial systems, so a higher level of technical expertise is required to install and manage them. However, users are likely to better understand and appreciate how intrusion detection systems work, as well as their strengths and limitations. In addition, we review Tripwire
[C22], a tool that, like Network Flight Recorder, comes in both a public domain version and a commercial version.
Shadow [R35], [B55-b] uses what it calls
sensor and analysis stations. Sensors are usually located
at important points in the network, such as outside a
firewall, while the analysis station is located inside the
firewall. Sensors extract the packet headers and save them
to a file. This is read hourly by the analyst station,
which then performs the filtering operation and generates a
second log file. The Shadow philosophy is not to provide
alerts when events are identified. This approach was
motivated through experience with other ID systems, where
many alerts turned out to be false alarms and were
distracting and annoying.
The sensor station uses the libpcap utility developed by the Lawrence Berkeley Laboratories Network Research Group
[R37] to provide a basic sniffer capability. The station does not preprocess the data, thus preventing an intruder from checking what is done with the packets.
Major support for analysis is provided by tcpdump
[R37] through which packet filters are defined and executed. However, some intrusions were difficult to detect with tcpdump filters, particularly those involving infrequent probes. For these types of events, Shadow provides a Perl-based tool, one_day_pat.pl, as part of its kit. This allows one to scan for low-frequency patterns that may occur in more than one log file. Filters can be simple or compound (Boolean) collections of simpler filters. An example of a simple filter is tcp and dest port 23. This simple filter selects packets with the TCP protocol and destination port 23 (i.e., telnet).
The analysis station uses a Web-based interface to display information from the sensors, or to display the results of filtering operations on the raw data. Shadow runs on many UNIX systems and on open source systems like FreeBSD or Linux.
Network Flight RecorderTM (NFR) is an ID system that was previously available in both a commercial version and a public domain version. The latter was freely available until quite recently and is the focus of this section
[C2],
[R38]. NFR explains the reasons for the change in policy in part:
NFR has discontinued access to the old source code Research version. This was necessary as the Research software was not as capable as our commercial product and users thought the Research version was our commercial offering.
NFR plans to make the commercial product available to researchers, though apparently not in source form. We retain the discussion below because we believe that much of it is applicable to the commercial version. Readers should contact NFR for additional information.
Like Shadow, NFR uses a modified version of libpcap to promiscuously and passively extract packets off the network (this version can also extract packet bodies). However, NFR goes beyond simply analyzing headers as it reassembles TCP streams. Data collection and analysis are usually performed on the same platform outside the firewall. However, copies of NFR can also be placed at strategic internal points to detect potential insider threats. Above the packet extraction level, a decision engine filters and reassembles packets.
NFR includes a complete programming language, called N, designed for packet analysis. Filters are written in the language, which is compiled into byte-code and interpreted by the execution engine. Through programs written in N, pattern matching is performed to allow packets to be reassembled, as one example.
The functions alert and record are used to extract data after the filtering operation and to support back ends. The alert function can send events via email or fax. The record function tailors the data into formats required by the various backend analysis modules. The developers originally thought there would be many small, special-purpose backends, but it turns out that a smaller number of multi-purpose backends were developed instead.
Histogram and list are two primary examples of multi-purpose backends. Histogram provides a facility for capturing data in a multi-dimensional matrix. Totals of relevant categories are accumulated in the cells of the matrix. Alerts can be generated based on the absolute numbers or relative frequencies within the cells. The list backend records chronological records, thus providing a level of detail (at the expense of storage) not provided by the histogram function.
NFR also provides query backends that allow you to analyze the data. Query backends were designed so as not to degrade the performance of the execution engine since this could lead to dropped packets. Query backends have their own CGI interface. Also, query backends provide graphical capability to allow data to be viewed more easily.
TripwireTM
[C22] is a file integrity assessment tool that was originally developed at Purdue University. Like NFR, it comes in both public domain and commercial versions; the public domain version is available in source code for UNIX systems. Tripwire is different from most other ID tools in that it detects changes in the file system of the monitored system rather than looking for suspicious activities, per se.
Tripwire computes checksums or cryptographic signatures of files. If these signatures are computed for a file system that is known to be in a secure or safe configuration and stored in such a way that they cannot be corrupted, for example, offline or on a write once medium, they can be compared with a subsequent recomputation of signatures for the same file system to determine which, if any files have changed. Tripwire can be configured to report all changes in the monitored file system or only those of interest to the administrator. For example, it can check if system binaries have been modified, if syslog files have shrunk, or if security settings have unexpectedly changed. It can be configured to perform integrity checks at regularly scheduled intervals and provides systems administrators with the information they need to implement recovery if tampering has occurred. Of course, recovery requires that appropriate backups of the material that has been compromised be available.
Tripwire can facilitate recovery as well as detection¾a feature not present in most ID systems. Tripwire's approach is independent of the type of exploit and it is not necessary for the exploit to be identified for recovery to be made, however, Tripwire will not detect an intrusion that does not modify monitored files and the detection will not be made until the next time Tripwire is run. Unscheduled Tripwire runs are useful for damage assessment after a suspected or confirmed intrusion.
Tripwire comes in both commercial and free versions. The current commercial release is version 2.X for a variety of UNIX platforms and Windows NT 4.0. The binary version of 2.0 for Red Hat Linux 5.1 and 5.2 is available for free. The academic source release of version 1.3 is also free and represents the state of the program as of 1992.
According to the vendor, all versions of the 2.X Commercial Release features include
Additional features are available for HP/UX and IBM/AIX platforms.
The primary requirements for network intrusion detection products (besides intrusion detection) are to make suspicious network activity visible, provide an alert about the activity, and offer a capability to stop it if possible. On the surface, commercial and government requirements are the same; however, they generally differ in scope and focus.
In February 1999, the Department of Energy, the National Security Council, and the Office of Science and Technology Policy sponsored a workshop titled "Detection of Malicious Code, Intrusions, and Anomalous Activity." The workshop was attended by participants from commercial and government sectors. A breakout session was formed to focus on network intrusion detection. It identified the following as requirements for ID products that will NOT be met by the commercial market:
Both commercial and government organizations require ID systems to detect network intrusions and provide visibility, but the scope and focus for these requirements is different.
While businesses are interested in protecting their networks and information, they are primarily interested in making a profit. Companies are motivated to purchase ID products as they become aware of the threat from network compromise, the possibility of losing critical proprietary market data or products, and the possibility of being liable for downstream damages to any trusted networks.
A second motivation is to ensure they have pursued best practices to keep from being liable to companies with which they have trusted network relationships or to stockholders in the event of loss of critical data or products. Given the bottom-line objective and the knowledge that best practices will keep them from being too liable, it is logical to surmise that the commercial market in the near term will be content with the current state of network intrusion detection technology.
Like businesses, the Federal government is interested in protecting its networks, but the primary concern is not in making a profit, but in protecting national security. Given this very important difference, governmental organizations requirement to detect intrusions has a much broader scope then the commercial requirement to detect intrusions. For the government, the capability to detect network intrusions is driven by the possibility of intruders (possibly sponsored by another country) breaking into government networks. All of the experts at the February 1999 workshop agreed that the resources and talents of a state-sponsored intruder would definitely exceed the current best practices of network intrusion detection products.
Another important distinction between the commercial and government requirements is focus. Businesses focus on gaining a picture suspicious network activity so that it can be stopped, whereas the government's focus may be to discern the intent of the person responsible for the activity. In special situations, the government may choose to collect information for intelligence purposes or for law enforcement. Businesses do not have this requirement and subsequently the vendors of best practice products available in today's market are not driven to develop this capability. Commercial ID products are designed to collect the minimum amount of data to detect suspicious activity; they are not primarily designed to collect additional data that government may need.
Government requirements alone (as articulated above) identify the need for continued work on GOTS network ID products, but there are additional arguments in favor of GOTS. The February 1999 workshop identified that the commercial market will not (on its own) develop objective evaluations of ID products. Currently, there are no standards for evaluating network intrusion detection products, so consumers must take vendor's claims at face value. This may suffice for companies purchasing ID products to ensure they have implemented best practices (and thereby limited their liabilities) but governmental organizations required to protect national security must know both what the product does and how it works.
Another issue arises because commercial ID products can be purchased by anyone. It is possible that an intruder who knows which commercial ID product is deployed by the government could purchase the same product and learn how to defeat it. This risk is minimized by using GOTS software.
The government has network intrusion detection requirements that current COTS intrusion detection products can NOT meet. Also, the commercial market will not drive the development of the COTS intrusion detection products to meet the government requirement to protect against the nation-state-sponsored intruder, or provide the data needed by intelligence organizations or law enforcement. Given these government requirements, it is essential that the government continue to develop GOTS intrusion detection products.
CIDDS, the Common Intrusion Detection Director System (also knows as CID Director), is a dedicated hardware/software/operating system platform supporting the Air Force Information Warfare Center's (AFIWC) Intrusion Detection Tools (IDT) program. AFIWC is the U.S. Air Force Office of Primary Responsibility for the IDT program. Within AFIWC, the Air Force Computer Emergency Response Team (AFCERT) is charged with the responsibility for day-to-day administration and network security operations involving the IDT program.
CIDDS receives near-real-time connections data and associated transcripts from Automated Security Incident Measurement (ASIM) Sensor host machines and selected other intrusion detection tools (see below). It stores this data on a local database and allows for detailed (local, regional, or theater-wide) correlation and analysis by human analysts and other automated tools.
The CID Director software consists of a suite of compiled C and C++ programs, compiled Java language programs, Oracle SQL functions, procedures, and scripts, an Oracle database structure, Borne shell scripts, and configuration files which together communicate with ASIM Sensor system machines to receive connections information from the Sensor hosts. The Director stores this information in a local Oracle database, and through the use of a user-friendly graphical user interface, allows the user to correlate, study, and analyze indicators of potentially intrusive, malicious or unauthorized events occurring on Air Force networks. Various uses for this data include
CIDDS also incorporates the ability to play back real-time connections data for keystroke-by-keystroke analysis.
CIDDS provides the ASIM system with a centralized data storage and analysis capability. The Director receives data from various Sensors that are monitoring and reporting on Air Force networks. These networks may be homogenous or heterogeneous, serving similar or diverse operational or support missions, and at various organizational levels. CIDDS is the central data collection, correlation and analysis point for all these network systems.
Plans for future development of the Air Force Intrusion Detection Tools system include a number of CID Directors positioned at various levels throughout the Air Force command structure, with the plan that all these Directors forward essential information to a common enterprise database in the AFCERT.
Each CID Director host machine is dependent upon the ASIM Sensor host systems reporting to it for the accuracy and timeliness of the data it receives and processes. The Director makes use of ASIM Sensor data to provide near-real-time reporting of events and provides a mechanism to review connection information in support of incident detection and event correlation. The CID Director, combined with the ASIM Sensor, provides the capability to proactively and automatically enforce Air Force network security policy and to protect Air Force networks from malicious, intrusive, or unauthorized activities.
The ASIM Sensor software consists of a suite of compiled C code and Java language programs, Bourne shell scripts, and configuration files which together capture, filter, and analyze ethernet and Fiber Distributed Data Interface (FDDI) data packets. ASIM Sensor is essentially a promiscuous data packet sniffer and analyzer. The ASIM Sensor software monitors network Internet Protocol (IP) traffic such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and Internet Control Message Protocol (ICMP).
In a typical ASIM Sensor system software installation (for normal network monitoring operations), both modes are started when ASIM is installed.
Real-time ASIM uses the same software engine as batch-mode ASIM to collect network traffic. Real-time ASIM identifies strings and services that could indicate attempts at unauthorized access at the time it occurs, and immediately spawns an alert process at the Sensor host and sends a real-time alert to its associated Director. Real-time alerts typically contain basic information on a specific activity. Additional information can be viewed by generating transcripts which are keystroke-by-keystroke records of the detected alert.
Batch-mode ASIM Sensor collects network traffic (data) for a configurable time period, usually 24 hours. After collection of the data, the data is analyzed by the software to identify indicators of suspicious activity. The ASIM Sensor software generates transcripts of collected data for an identified connection that could indicate someone attempted or performed unauthorized activity. Data can be viewed at the local ASIM Sensor site or sent to a CID Director machine or central office (AFIWC/AFCERT) for review and analysis.
The data collected by ASIM Sensors at AFCERT-monitored sites is encrypted and sent to ASIM Central (AFIWC/AFCERT) each day for review by a human ASIM analyst. The analyst determines whether the activity ASIM Sensor identified is malicious, unauthorized, or normal authorized activity.
This section characterizes the current market view of ID systems. It also examines ID capabilities and looks at the ways in which organizations use ID systems. The section closes with a statement of the market condition as viewed by the authors.
A great deal is being written about the capabilities of ID systems. One of the inherent difficulties for a consumer of a new, immature technology is how to interpret generalized claims of capability as they apply to the consumer's needs and how to separate hype from reality. This section includes statements from several authors in response to the question, "What can intrusion detection systems do?" These assertions are part of the market environment in which ID tools are being considered for purchase and use. They also serve to set consumer expectations.
A number of capabilities are claimed for ID products in an ICSA paper titled "An Introduction to Intrusion Detection and Assessment"
[B23]. They can
In a 1998 Computer Security Institute round table discussion of ID experts, the following perspectives were offered
[B15]:
A recent (July 1999) Information Week survey
[S30] of 2,700 executives, security professionals, and technology managers from 49 countries concludes that more companies are using intrusion-detection systems that scan the network for trespassers and alert IT personnel in real time if intruders are discovered. This year, 37% of survey respondents reported using intrusion detection products, up from 29% last year. And every company that said it uses intrusion detection systems discovered unwelcome outsiders prowling in their systems.
"That 100% of users were able to catch intrusions with [intrusion detection
systems] is a testament that they actually work," says
Pricewaterhouse/Coopers' Lobel. The effectiveness and growing ease of use of
intrusion detection systems has helped fuel their use. "People are looking for
less manually intensive and less reactive tools so they can deal with
incidents in real time," Lobel adds. The tools are designed to help IT
managers save time, which is important because lack of time was cited as the
main barrier to implementing improved security.
The survey includes a numerical breakdown of responses to the question "How have you learned about your security breaches?"
The results are shown in Table 2-2. The table, in which multiple responses were allowed, implies that reliance on ID systems is growing proportionately more than in any of the other categories.
The current IDS market position is further supported by the 1999 CSI/FBI Computer Crime and Security Survey
[B94] of 521 security practitioners in US corporations, government agencies, financial institutions, and universities. It states that "the use of intrusion detection systems (IDS) rose from 35% in 1998 to 42% in 1999."
The ICSA/SAIC 1999 security survey (745 respondents)
[S33] provides wide-ranging information
about industry's attitude to computer security. Responses to one of the questions provides insights into what market sectors are implementing ID systems. Table 2-3 summarizes this data.
With respect to what market sectors will be purchasing ID systems within the next year, the following were the responses
[S33]:
From these various surveys, one could conclude that ID technologies are becoming an accepted part of many organizations' information security tool suite. The concern of the authors is that such organizations are viewing these tools as solving a class of problems. The problems are not fully understood and the solutions are likely inadequate and possibly incorrect. This can create a false sense of confidence in the degree to which intrusions against an organization's critical assets are detected.
Through our own experience and in discussion with
technology experts and market analysts, we observe that the
current market condition of commercial ID tools and
technologies exhibits a growing bandwagon effect. Each
organization is referencing others in their peer group or
market segment and comparing what they are doing with what
the competition is doing. If an organization views itself
as taking security protection actions (such as deploying an
IDS) that are equal to or slightly better than an
organization that it considers its peer, that is good
enough. At the decision-making level, there appears to be
little to no regard for what ID systems can actually do nor
any appreciation for what they cannot do and should not be
relied upon to do. Management's priority appears to be to
ensure that they can demonstrate that they have exercised a
standard of due care in the event of any legal action. We
believe that the vendor community is marketing to this
condition through the product claims they make. It is our
observation that many of the claims made in Section 2.2.1 and the
statements of survey responders in Section 2.2.2 cannot be
demonstrated in actual IDS use or, at the very least, are
somewhat overstated and misleading.
This section describes the CERT/CC® IDS team's experiences in installing, configuring, and
operating several network-based ID tools on the operational CERT network, both
inside and outside of the external firewall protection mechanisms. Our intent
was to do what ID tool developers advised in their documentation and learn,
through experimentation and use, how well each tool did what it advertised.
From the knowledge we gained, we formed additional judgments to support our
determination of the state of current practice. There was no intent to
evaluate a specific tool or to compare one tool with another, nor was there
any intent to perform formal testing of the frequency of false alarms.
We installed each ID tool in the CERT DMZ, which is located
between the CERT private network and the CERT Internet connection. By choosing this location, we were able to monitor all traffic between the CERT private network and the Internet, between the DMZ and the Internet, between the private network and the DMZ, and between machines on the DMZ. Every machine in the DMZ is plugged into a single switch. We set one port of the switch to mirror the traffic of all of the other ports. We plugged each IDS into this port via a hub and a cable engineered to allow traffic to flow one-way only (from the switch to the IDS) as shown in
Figure 2-1.
There are two problems with this approach: it isn't scalable, and it is possible to lose packets since the output from the monitored switch ports may be greater than can be handled by the port to which the traffic is mirrored (where the IDS resides). To address these problems, we could have monitored more selectively, i.e., the ports to which the inside and outside firewalls are connected. In doing so, we would have only lost the ability to monitor traffic between machines in the DMZ. However, by monitoring the switch (and all ports), we were able to tell if packet loss was a problem and were more likely to detect more events of interest. We do not recommend this configuration; however, it does demonstrate some of the trade-offs to determine an appropriate location for an IDS and what traffic to monitor.
We installed ISS RealSecureTM (Section 2.1.2.5), Cisco NetRangerTM (Section
2.1.2.3), Network Flight RecorderTM (Section 2.1.3.2), and Shadow (Section 2.1.3.1).
The policies governing our operational environment and the customized nature of the CERT infrastructure did not permit experimenting with host-based ID systems on the UNIX hosts that serve as our primary public server systems. We have tailored log filtering and analysis tools to meet our intrusion detection needs and host-based ID systems do not appear to add sufficient value for the cost and effort (primarily staff resources) required to deploy them in this environment.
This section describes our observations during ID tool installation and configuration.
The most important decision in installing an IDS is identifying its location(s). This decision has security implications. Most of the ID systems we examined require two interfaces: one insecure interface to perform monitoring and one secure interface to communicate with and manage the IDS.6 We do not believe this is a reasonable approach in a production network, specifically the use of an insecure interface for monitoring, when the IDS can both read from and write to the network because there is a potential for the platform hosting the IDS to serve as a bypass for the firewall if it is compromised. If an IDS is operated with an unsecured interface, it is necessary to trust the IDS as much as, for example, the firewall and pay the same level of attention to it as the firewall requires. The approach was to secure the IDS hardware and software as much as possible and use a secure protocol such as ssh to communicate with and manage the IDS. In our judgment, the best approach is to use two interfaces as first described but ensure, through hardware connectivity and control, that the monitoring interface can read traffic traversing the network but can not write to it. The drawback to this approach is that the IDS's capabilities for automated responses that require access to the network such as killing connections and reconfiguring firewalls cannot be deployed. We do not see this as a problem for us as we would not invoke any of the provided automated response capabilities based on our lack of confidence in the ID system's ability to distinguish between actual intrusions and false alarms.
We found that commercial ID tools, in general, required less time to install than public domain tools. It appears that vendors have spent considerable time making the installation process as painless as possible. Organizations with large infrastructures should carefully consider 1) the time required to install and maintain a public domain ID solution, and 2) if the public domain solution will scale.
None of the tools we examined had an understandable, easy-to-use configuration interface. However, the commercial tools did employ graphical interfaces where the public domain tools did not. Our primary objection is that all the tools require the operator to tune signatures individually. If the signatures were grouped into similar categories and the operator was able to tune a group at a time, the configuration process would be simplified. As an example, RealSecure documentation provides an assurance level for each signature (no false positives, few false positives, high false positives). It would have helped if we had been able to turn off ID checking for all signatures except those possessing the "no false positives" assurance level. The only way to accomplish this was to configure each signature to be "on" or "off" based on its assurance level.
Among the tools we examined, we found no indication of any integration between vulnerability scanners and configuration interfaces. The configuration process could be made more simple if detected vulnerabilities are used as the basis for the inclusion of IDS signatures, i.e., the IDS only checks for those vulnerabilities that are present and relevant to the installed system and application software (as contrasted with including signatures for software that is not present). Most, if not all, commercial vendors that sell ID tools also sell vulnerability scanners, so this level of integration could be accomplished.
The majority of ID systems we examined appeared to provide good capabilities for enhanced network monitoring. However, most of the systems (both commercial and public domain) are not advertised from this perspective. Tools that are so advertised often don't provide the flexibility to examine specific packets as ID systems do.
Specifically, ID systems can monitor the following:
During the course of experimenting with the ID systems, we discovered one event that couldn't have been resolved without using one of these systems. Late one evening, a service simultaneously died on a number of machines in our DMZ. We looked through system logs and records but were unable to find the cause. We were able to locate the time that the services died and examined the tcpdump logs generated by the Shadow system
[B55-b]. We identified the packets in the logs that appeared to have killed the services. We found the remote machines that sent the packets, and were able to get them to send the bad packets again to verify the results. With additional analysis, we discovered that the remote machines were running a version of software that produced output our local services couldn't handle. We were running the latest software version of our local services. Since the bad packets killed our services without doing any permanent damage, we were able to temporarily solve the problem by writing a script to monitor our local services and restart them when they died. We reported our finding to the software vendor and they fixed this problem. We may not have found the problem if the IDS wasn't running, and we definitely wouldn't have found it as quickly.
We experienced two major shortcomings related to detecting intrusions using a signature-based analysis approach:
Each category is explained below.
Underlying Issues with Signature-Based ID
Basing an ID analysis approach on the detection of known behavior patterns that exploit known vulnerabilities results in addressing symptoms, not root causes. A better use of an administrator's time and resources is to minimize or eliminate the vulnerability through the application of patches or other security measures. This focus on symptoms rather than root causes is further exacerbated by the use of signatures that detect the exploitation of known vulnerabilities in software that may not even reside on the network or host where the IDS is deployed (e.g., detecting UNIX-based patterns on a solely NT-based network). Integrating vulnerability scanners and ID systems could offer great benefits; the scanner's output could serve as input to the IDS and aid in determining what to detect (signatures selected on the basis of scanned vulnerabilities) and what to ignore (signatures eliminated due to the absence of vulnerabilities).7
The vendor products that we installed did not provide sufficient supporting data (such as raw packets) to test for events they claimed to detect. They also did not provide the signature algorithm that reveals how the determination of an event is made. In our judgment, this lack of information causes system administrators to spend excessive time evaluating reported false alarms, with the result that real attacks will ultimately be ignored. Given these limitations, every event requires extensive manual investigation to see whether it is real or a false alarm. For example, when a Web server exploit alarm is reported on the IDS console, the operator must look through the Web server logs to see if something actually happened. If the IDS provided all of the packets (including the responses) as supporting information, it would be fairly straightforward to see what actually occurred. Similarly, if the signature algorithm were available, the operator could determine if the signature includes checking the responses from the Web server.
IDS products based on current signature-based analysis approaches do not provide a complete intrusion detection solution but do produce useful results in specific situations and configurations.
We would like to see ID products provide test data to better assess the accuracy of events that are detected. We strongly support the growing community interest in open-source signatures. At this point in time, Network Flight Recorder (NFR) is the only commercial product that we would consider implementing given that the source code (n-code) for the signatures is readily available. NFR offers fewer signatures than other commercial products; however, this is being remedied in a recent agreement between the developers of NFR and L0phtTM
[R88].
It then analyzes that traffic to identify suspicious activity. There are two modes of operation for ASIM Sensor: 1) batch mode, and 2) real time.
2.2.2 Recent Survey Results
Figure 2-1: IDS evaluation setup
Figures in this file are displayed in a separate browser
window. This window will remain open to display figures in this file,
although it might be hidden behind other browser windows.
2.3.4 Shortcomings
[Title Page]
[Abstract]
[Figures]