Software Engineering Institute Carnegie Mellon

State of the Practice of Intrusion Detection Technologies

4 What Are the Organizational Issues?

It is as important to have effective security policies in place as it is to have the latest technology if one wants to secure a network against intrusions. Effective policies will help to ensure that, among other things, security managers are adequately trained, information derived from ID systems is interpreted correctly, and appropriate actions are taken when an intrusion is identified. These issues are particularly important since, unlike virus detection systems, ID systems require care and attention for successful operation.  

 

4.1 Barriers to Effective Security

There are many reasons why organizations fail to provide effective security. Recognizing these may help an organization develop more effective security strategies. Recent surveys summarize some of the reasons that organizations give for their failure to act in this area. The ICSA/SAIC 1999 security survey (745 respondents) [S33] posed the question, "What is the SINGLE greatest obstacle to achieving adequate information security at your organization?" Replies fell into the following categories.  

 

Table 4-1: Barriers to ID system adoption - 1
Obstacle
 		Percentage of organizations
Budget constraints
 		29
Lack of senior mgmt support
 		14
Lack of employee training/end user awareness
10
Lack of competent IS personnel
 		9
Lack of internal policies
 		8
Lack of centralized authority
 		8
Technical complexity
 		6
Unclear responsibilities
 		4
Lack of good security products
 		3
Other
 		9

The InformationWeek 1999 global security survey (2,700 respondents in 49 countries) [S30] states the following in response to the question, "Which of these is the most significant barrier to effective security in your company?"  

 

Table 4-2: Barriers to ID system adoption - 2
Barrier
 		Percentage of organizations
Lack of time
 		17
Complexity of technology
 		16
Pace of change
 		11
Lack of management support
 		11
Poorly defined policy
 		10
Capital expense
 		8
Lack of dept/group cooperation
 		8
Lack of training
 		8
Lack of qualified staff
 		6
Labor expense
 		5

A survey [S36] of 1850 computer-security experts and managers taken by the SANS Institute at conferences held in May 1999 identified "Seven Top Management Errors that Lead to Computer Security Vulnerabilities." They are

While these responses address the broader topic of information security, they are clearly germane to a specific security technology such as intrusion detection systems. This section describes several of these issues as they relate to the use of ID technology including

  • understanding the threat as a prerequisite for determining effective means for protection

  • the critical role of management sponsorship and support

  • setting the appropriate policies, procedures, and mechanisms for their enforcement

  • understanding the full IDS life cycle and what is required to support it

  • heightening awareness and regular training

  • factors to consider in the decision to make, rent, or buy ID staff capability

  • managing expectations of what ID technology can and cannot do
 

 

4.2 Understanding the Threat

Before an organization makes an investment in security technologies, it is important that it understand what assets require protection as well as the real and perceived threats to those assets. Threats can be characterized by the type of attack, the category of the attacker in terms of their capabilities, resources, and goals, and the organization's tolerance for loss of the asset that is being protected. Loss can be characterized as a loss of confidentiality, availability, or integrity.

An attacker can have one or more objectives in attacking a computer network. Several of these objectives are identified in Table 4-3, which provides an approximate correlation between types of attacks and the objectives of attackers. In the table, information modification connotes the clandestine change of data (i.e., so that the changes are not noticed) while information corruption renders the information unintelligible.

These modes of attack may reflect different attack signatures; this may imply different intrusion detection strategies. For example, information retrieval is likely to be performed using a stealthy attack, while for information corruption, stealth may be less important than speed. A bank, concerned with illegal financial withdrawals, likely considers detecting stealthy attacks very important.

Hence, the chosen intrusion detection approach depends, to some extent, on the objectives of an organization's likely adversaries. A paper titled "Who's stealing your information?" [B51] describes who is involved in illegal information retrieval, a current problem causing a great deal of financial loss, and likely to become much worse. The InformationWeek survey [S30] suggests that the problem of insider attack is perceived to be decreasing relative to the problem of attack from external intruders and terrorists. Determining whether the potential attacker is inside or outside of the organization's infrastructure has a bearing on the type of IDS you select.  

 

Table 4-3: Intruder motives
Objective Denial of service (loss of availability) Information retrieval (loss of confidentiality) Information modification or corruption (loss of integrity)
Curiosity
X
Vandalism X
X
Revenge X
X
Financial gain

X
Competitive advantage X X X
Intelligence gathering
X
Military gain X X X
 

 

4.3 Management Sponsorship and Support

Often the most significant obstacle to the success of an information security improvement initiative is lack of management support1. The information in Table 4-1 and Table 4-2 is consistent with our experience at the Software Engineering Institute in implementing improvement initiatives, including those focused on security improvement. Managers have many goals to meet, and they must often make compromises between them. In today's competitive environment, companies are focused on achieving market share and minimizing product cycle time, achieving financial and other performance objectives, and worrying about mergers, acquisitions and reorganizations. In such a climate it is no surprise that security concerns are often relegated to a lower priority. Only when some significant security breach has occurred that affects a high priority business objective (such as preserving the organization's reputation) is the importance of security elevated.

Management sponsorship is demonstrated by

  • visibly being supportive of efforts to improve information security

  • encouraging staff to communicate security concerns at all levels of the organization

  • follow-through on the concerns that are expressed

  • the sustained allocation of sufficient resources to accomplish security improvement initiatives (budget, staff, time)

  • the existence of an organizational security policy and procedures that are documented, understandable, not open to misinterpretation, not overly burdensome or restrictive, and that cover the range of topics required to meet the organization's security objectives

  • visible, demonstrated enforcement of the policy including consistent application of sanctions for non-compliance

Unless there is significant Chief Information Officer and Information Security manager-level sponsorship and support for the deployment of an IDS and all that it implies, the successful operation and use of this technology will be short-lived, sustained only by the interest of those internal champions who believe in its benefit, until the next high-priority item requires their attention.  

 

4.4 Policies, Procedures, and Mechanisms for Their Enforcement

Establishing an information security policy so that it reflects an organization's business goals and enacting that policy so that it is a normal, accepted part of day-to-day operations are difficult at best. The business goals need to be well articulated both initially and as they change. The security implications of each business goal need to be identified and transformed into organizational security requirements. Meeting these requirements needs to be folded into the objectives and incentives of the responsible managers. Once the requirements are well understood, a meaningful security policy (and supporting procedures) can be developed with reasonable confidence that the content and intent will be enacted.

Charles Cresson-Wood [B131] provides a comprehensive description of how to go about developing an information security policy and of the content items that should be considered. There are numerous additional references that address this topic. For more specifics on policies related to detecting and responding to intrusions, refer to the SEI reports Detecting Signs of Intrusion [B98], Preparing to Detect Signs of Intrusion [B116], and Responding to Intrusions [B123]. Developing the content is the easy part; making it real in the behaviors and actions of the organization's managers and staff is more difficult.

Effective means for security policy deployment include

  • disseminating the security policy to all employees and requiring their agreement to follow it, demonstrated by their signing off on the policy

  • providing policy information during security awareness training. Initial training should be given to new employees and periodic refreshers should be given to all staff, including managers. The fact that senior managers and executives are willing to make the time to attend such training sends a strong message of support and importance (see Section 4.5).

  • visibly enforcing the policy, including consistent application of sanctions for non-compliance

  • regularly scheduling security policy topics during management reviews and staff meetings. Security policy reviews should identify and update areas that are not working well, reflect new business directions, and examine policy effectiveness in meeting organizational security requirements and their corresponding business goals

  • involving all stakeholders who are affected by any security policy development or update. Even though this takes more time and resources up front, it saves significantly in deployment by achieving the necessary buy-in and commitment required to ensure success.
 

 

4.5 The IDS Life Cycle

An organization needs to fully appreciate the commitment required before deploying an ID system. Otherwise, the project runs the risk of wasting time, money, and staff resources in the initial phases of the life cycle. Sufficient resources must be committed at a level appropriate to cover all phases of IDS use. The life cycle phases that are described in this section include

  • deploying a defense-in-depth or layered security architecture

  • evaluation and selection

  • deployment

  • operation and use

  • maintenance

 

 

Defense in Depth

The following steps should be taken whether or not the decision is made to deploy an IDS:

  • eliminating as many known vulnerabilities as possible (applying patches, securing or hardening configurations)

  • deleting unnecessary services

  • using one or more firewalls for limiting access (both externally and internally, if required)

  • implementing access control and user authentication mechanisms such as encrypted or one-time passwords, smart cards, and access control mechanisms at the network, system, application, and file levels

  • using internal and external monitoring tools to detect suspicious events

  • using WORM (write once, read many) or append-only log devices and files to prevent intruders from covering their tracks by deleting log records

  • managing and monitoring modem connectivity to ensure unauthorized modems are not being used to circumvent firewalls

  • verifying configurations through systematic "self-attack" or penetration testing

  • using an integrity checking tool such as Tripwire [C22] to detect policy non-compliance

  • using vulnerability scanning tools and proactive system administration for reducing vulnerabilities

  • using virus detection and eradication software

  • conducting on-going end user awareness training on operating securely, what constitutes suspicious behavior, and how to report it

  • conducting on-going security training for system and network administrators
 

 

Evaluation and Selection

This phase of the IDS life cycle first involves determining if one will install and manage the IDS in-house or involve outside agents in these activities. Section 4.7 provides some pros and cons with regards to making this decision. If one decides to perform the implementation in-house, then selecting the IDS that best satisfies an organization's requirements and goals is essential. This involves collecting and analyzing all relevant vendor information, asking hard questions, potentially performing IDS testing, and making an informed selection decision.

A summary of topics to consider when evaluating ID systems includes

  • appropriate approaches to intrusion detection

    • host and/or network capabilities

    • signature and/or anomaly-based approaches

  • detection and response characteristics

    • accuracy of ID system's diagnosis (frequency of false alarms)

    • ability to customize signatures

    • proprietariness of signatures

    • ability to suppress responses from signatures that exhibit high false alarm rates

    • ability to give confidence level with diagnosis

    • providing guidance in response to attack

  • performance

    • speed of attack detection

    • robustness of IDS under attacks on itself

  • ease of use

    • effectiveness of user interface

    • ease of installation

    • the need for specialized hardware for operation

  • non-technical issues

    • cost of system, including base software, installation and operation

    • company reputation (including stability, longevity, responsiveness)

    • effectiveness of documentation and training

    • rapidity of signature update from vendor

More detailed information that expands each of these points can be found in the paper "Tough Questions for IDS Vendors" [B14] and in Appendix F. This appendix contains excerpts from a paper by Amoroso and Kwapniewski titled "A Selection Criteria for Intrusion Detection Systems" [B57].  

 

Deployment

Prior to operational use, there are a number of IDS topics that need to be considered. These include

  • locating the ID sensors to protect those network resources where the most valuable assets reside, rather than trying to protect every resource on the network

    • requires ranking the value of assets according to their priority before sensor
      placement

  • installing and configuring the IDS to reflect the security policies (Section 4.3)

  • establishing ID policies with supporting procedures that

    • define attack signatures and anomaly-based profiles

    • identify procedures to collect and analyze intrusion data, and respond to the intrusion (see Section 5.2.2).

    • specify conditions under which automated response is permitted and how the outcome of such a response is monitored to ensure the appropriate action is taken

    • provide guidance on when to escalate attack information to management

    • guide the collection forensic evidence

    • address legal issues

  • possibly installing a honeypot [C21-3]

  • establishing initial anomaly-based profiles

  • establishing initial attack signatures (whether provided by the vendor, adapted from the vendor's signatures, or developed in-house)

  • accounting for the fact that ID systems cannot typically report on application-generated security events as they are generally monitoring only operating system or network events. Certain application events where knowledge is available (e.g., http attacks) can be included.
 

 

Operation and Use

In the Operation and Use phase, consider processes, procedures, and mechanisms for

  • allocating roles and responsibilities for analyzing the results that an IDS produces and acting on those results

  • actions that need to be taken when an alert occurs

  • identifying conditions under which automated response is permitted and how the outcome of such a response is monitored to ensure the appropriate action was taken

  • establishing initial anomaly-based profiles

  • establishing signature-based signatures if these are not provided by the vendor or if the vendor's signatures need to be modified and expanded
 

Maintenance

In the maintenance phase, consider procedures and mechanisms for

  • updating anomaly-based profiles to reflect current user, system, and process behavior

  • updating signature-based signatures to reflect changing technology, security policies or other needs

  • replacing old versions of the IDS with new versions

  • maintaining awareness of ID technology improvements
 

 

Resources and Commitments

It is critical for an organization to clearly identify and assign

  • all roles

  • all responsibilities

  • scope of authority

in order to carry out each task in each IDS life cycle phase as described above. Once these have been determined, the organization needs to identify and allocate the required staff, funding, and resources for deploying an IDS, along with the priority that these tasks have in comparison with others. Deploying and using an IDS will be successful only if the required tasks are considered as part of an organization's normal strategic and operational planning cycles and the plans for which each manager is held accountable are reviewed.  

 

4.6 Awareness and Training

All users of an organization's information infrastructure need to become security conscious and receive periodic training if their behavior and actions are to reflect the organization's expectations with respect to security. Users include all executives, managers, staff members, business partners, vendors, contractors, and suppliers and, depending on how business is transacted, may even include customers. With training, assignment of responsibilities and authority can be made consistent with an individual's observed expertise and competence.

Security training should address the following topics:

  • the organization's information security goals, objectives, policies, and procedures including sanctions for non-compliance

  • secure use of information and computing resources

  • how to secure the information for which users are responsible

  • technical subjects such as appropriate use, password management (selection, protection, update), file access controls, expectations of privacy, user software installation, virus protection, remote access, encryption usage, and safe web browsing (specific topics should be selected from security policies and procedures)

  • the nature of suspicious behavior, attacks, and intrusions (including social engineering attempts) and recognition and reporting of such events

For a complex and evolving technology such as intrusion detection, standard approaches to training (e.g., stand-up or video presentations, computer-based training, and other forms of self-paced tutorials) may not be sufficient. This is due to the need for hands-on experience in analyzing data and tracking patterns of intrusive behavior. The dynamic nature of computer technology in general, and rapidly changing threats and tactics in particular, may stress traditional forms of mentoring and on-the-job training. In other fields where immediate and intuitive response is required, simulation plays a significant role.2 We believe that it could contribute in support of intrusion detection training [B43], [B75].  

 

4.7 The Decision To Make, Rent, or Buy ID Staff Capability

The term "make" indicates that the organization has hired and trained its own staff, so they are capable of selecting, deploying, operating, and maintaining an IDS. "Rent" is synonymous with "insourcing," which involves using another part of the organization (other than the one deploying the IDS) to provide the necessary staff expertise and resources, or bringing in consultants who act as part of the staff. "Buy," also known as "outsourcing," involves contracting with an organization different from the one deploying the IDS to provide all necessary ID skills and resources. In this last option, the service is usually provided remotely.

The information that ID systems produce can be extremely sensitive, which makes a case for building an organizational capability or insourcing from another business unit. However, it is increasingly difficult to attract and retain all of the necessary staff with the requisite skills to cover the full ID life cycle, particularly in light of the current market competition for people with this experience. In a CSI paper [B15], ID security experts, including Christopher Klaus of ISS and Marcus Ranum of NFR, see a growing trend towards outsourcing and make the following points:

  • Regardless of how sensitive the data is, [organizations] just don't know security very well. It all depends on how critical the data is and how security savvy the end user is. Most companies would do it themselves if they could. [Klaus]

  • My feeling is that most users won't want to deal with these issues [how much to record, how long to keep it, and how to present it to the end user], which are complex and expensive. They'd rather buy an IDS as part of a complete network access/security package, managed by an outside agency with a 24x7 operations center. [Ranum]

  • [Analyzing and acting upon an alert] takes a dedicated, experienced staff that sees these intrusions on a regular basis, knows how they work and more importantly, knows how to deal with them. The training and staffing requirements for this are just immense. Most companies don't have the capability, can't afford to build it, don't have time to build it, and even if they could build it, can't find the resources to build it with. [Curry]

  • The ''security skills gap'' has left most organizations with little ability to really understand security at this level of technical complexity. Therefore, I think you're going to see more organizations turn to outsourcing for network security. We often hear an initial position by a client that the corporation ''will not outsource security.'' We understand the logic and seldom raise the outsourcing issue. However, after they've seen the training requirements and the costs for 24 hour operation, they reconsider. One question often brings new light to the issue. Which would you trust more with your corporate network security? An employee who could be working for your competition next week or a service provider that is contractually bound to protect your corporate interests?

    This question usually leads to some interesting discussion. People outsource the monitoring of their home alarms. In fact, consumers have realized that home security alarm systems are of little value unless remote monitoring is involved. [Sutterfield]
 

 

4.8 Managing Expectations

Chief information officers and information system managers need to clearly set the organization's expectations regarding what the ID systems can and cannot do, particularly in light of the gaps identified in Section 3. The selection and deployment of an IDS need to be performed in the context of an overarching security architecture that reflects a layered approach to protecting an organization's assets as described earlier in this section. As a result, managers and staff should understand that an IDS has a role in protecting an organization's critical information assets but is only part of the information security solution, not a silver bullet. In fact, the topics in this section describe much higher priority security measures with greater cost/benefit advantages that should be adopted before considering the use of an IDS.  

 

 

 

1 One individual told an author of this report that he obtained management sponsorship by demonstrating how easy it was to break into his manager's confidential computer files. This approach is not necessarily recommended, but at least in this case, appears to have been effective!

2 Good analogies are learning to fly a plane or control a nuclear power plant. One must understand the theory. However, hands-on experience (as can be provided by a simulator) is essential before one is considered to be proficient.
 

 

[Title Page]     [Abstract]     [Figures]     [Acknowledgments]     [Executive Summary]     [Preface]     [Section 1]     [Section 2]     [Section 3]     [Section 4]     [Section 5]     [Appendix A]     [Appendix B]     [Appendix C]     [Appendix D]     [Appendix E]     [Appendix F]     [DTIC page]     [PDF file]