Avoiding the Trial-by-Fire Approach to Security Incidents



Moira West Brown

This library item is related to the following area(s) of work:

Security and Survivability

This article was originally published in News at SEI on: March 1, 1999

In previous issues of this column, we have written about the importance of considering security from product design or procurement through deployment and use. Being proactive about security is critical to mitigating your security risk. However, having good security measures in place won’t prevent you from suffering computer security incidents, so it is also important to be prepared and proactive about detecting and responding to such incidents when they do arise: Organizations may be proactive about fire prevention, but it is still necessary to have detection (e.g., smoke detectors) and response (e.g., sprinklers and fire extinguishers) in place to address a possible fire. Moreover, fire drills are conducted to ensure that the planned response approach is effective and understood by all involved. An organization’s approach to security must be just as comprehensive in its scope as fire prevention, detection, and response. This article will explore the range of options that exist in organizations today for detecting and responding to security incidents.

The trial-by-fire approach

Experience shows that most organizations don’t think about how to respond to a computer security incident until after they have experienced a significant one! This problem is common; many organizations have not assessed the business risk of having no formal incident-detection and response mechanisms in place. More often than not, organizations receive reports informing them that they are involved in an incident from some other part--rather than identifying the incident themselves!

The problem stems from a lack of organizational recognition of the need for a comprehensive security infrastructure. It is not until after an ill-prepared organization has suffered a significant security incident that the real business risk and impact of such an event becomes apparent. There may be some organizational or management perception that network and host security is something that the system and network administrators handle as a part of their day-to-day activities, or that security is handled by the organization’s firewall. Sadly this perception is often incorrect on both counts. The priorities of such staff are primarily focused on maintaining basic support and operation of the vast amount of computing equipment in place. Firewalls may prevent some attacks, but can’t prevent all attack types, and if not correctly configured and monitored they may still leave the organization open to a range of others. This approach, or lack of one, results in significant problems including:

  • not knowing if or for how long a network or systems have been compromised
  • not knowing what information is at risk, has been taken, or has been modified by intruders
  • not understanding the methods that the perpetrator(s) use to gain access to systems
  • not understanding what steps can be taken to stop the intrusion activity and secure the systems and network
  • failure to identify in advance any possible adverse effect that steps taken in response to an incident may have on the company’s ability to conduct business
  • not knowing who has authority to make decisions related to containing the activity, contacting legal department/law enforcement, etc.
  • delays in identifying and contacting the right people to inform them about the activity (both internally and externally)
  • no recognized point of contact in the organization that external or internal parties know to report to

The volunteer approach  

Some organizations have system and network administrators who are either interested or trained in computer security. Such individuals are better prepared to address security within their domain of authority--such as the machines in one department or operating unit, or the equipment on a given network segment. Within some organizations, various individuals may be working together to address security informally. This approach often stems from a group of individuals in the organization who see the need to address security even if the need is not recognized by higher level management. However, even then, having good people available doesn’t mean that the organization is prepared to respond. Depending on the scope of the overall volunteer effort, it is likely that even with intrusion-detection software in place in parts or the organization, serious network security incidents may still go undetected. Although this approach is a marked improvement over the trial-by-fire approach, significant problems still remain including

  • Serious intrusions may still go undetected.
  • Volunteers may be able to deal with the technical issues, but may not understand or have the information available to assess the business consequences of any steps taken.
  • Volunteers may not have the authority to apply the technical steps (e.g., disconnecting the organization from the Internet) or other actions they believe are necessary (e.g., reporting the activity to law enforcement or seeking the advice of legal counsel).
  • There may be delays in seeking and obtaining management approval to respond.
  • Volunteers have no "bigger picture" of the overall detection and response activity.
  • Volunteers may know in some cases who to contact internally, but anomalies may exist.
  • Other individuals in the company who identify a possible security incident may not be aware of the informal group and may fail to report to it.
  • An informal group is unlikely to have external recognition and support.

Company-supported approach

Regardless of the good intentions of technical experts or other staff members, the only effective approach to incident detection and response is to make it part of an organization-wide risk-management plan based on a foundation of management support at the highest level within the organization. Regardless of how such an approach is implemented, (whether by a geographically distributed or centrally located team, consisting of full- or part-time staff or supplemented with contract support), without management support, the effort will struggle to succeed. In addition to the foundation of management support, the empowered group must also be recognized internally and externally and prove its effectiveness, trustworthiness, and ability to all involved. Management authority and recognition are the foundation for success, but an effective detection and response service needs the trust and respect of the constituency served and others with whom the service will need to interact.

Teams established to address incident detection and response for organizations are known as computer security incident response teams (CSIRTs). Forming, staffing, and operating a CSIRT is not easy. However, if appropriately set up and empowered within an organization, a CSIRT can begin to gain the trust and respect necessary to address incident detection and response from a company-wide perspective. CSIRTs vary in structure, staffing, and the range of services provided based on the situation or need that they are trying to fulfill. Consider the need for a CSIRT in your own organization, whether it is company wide or just for your business unit or department. A recently published handbook, Handbook for Computer Security Incident Response Teams (CSIRTs), is now available to help an organization determine the scope and range of services for a CSIRT and provide guidance in forming operational CSIRT policies and procedures.

Advocating a company-supported approach

Making the transition from a trial-by-fire or volunteer response effort to a company-supported one is not easy. The most important and often the most difficult challenge is convincing management of the business need for an effective and empowered CSIRT as part of an overall risk-management approach.

Waiting for a serious security incident to occur within your organization to convince management of the need is not a productive approach, nor will it necessarily be successful. Even after suffering a serious computer-security incident in which hundreds of systems are compromised, some organizations still don’t recognize the need for a formal incident-response capability. I remember one case in which I contacted a multi-national company to inform them of information that indicated that an intruder was gaining access to the company’s corporate network through the Internet. As a result of the report, the company began to look at its systems and found that they had been seriously compromised for over six months. The company was able to identify many systems and internal networks that were compromised by the activity and the sensitive information available on those systems, but had no idea of the intruder’s motives nor the extent of the data that the intruder had copied or amended. A significant period of time elapsed and further compromises occurred before the organization established a CSIRT.

Another organization that was compromised by an intrusion took the step of reinstalling all of its systems from known good backups--losing two weeks of production effort in the process--as they could not be certain what data might have been tampered with by the intruder. In this case, malicious modifications to the application under development could have resulted in loss of life if the application had failed during use. The organization involved promptly established a company-supported CSIRT.

One of the most important factors to document is the associated business risk or loss of any incident. This information must be presented in a form that will help management understand that the problem is a business one and not a technical one. I recall one case in which technical staff had great problems in gaining management attention regarding ongoing intrusions. It was not until the intrusion data was presented by describing the mission of each system in question rather than providing its hostname and operating-system version that management paid attention. Volunteers should attempt to document and present to management the impact of known intrusions and recorded losses.

The influence of insurance

I learned of one situation recently in which a security officer compromised the home system of a manager as a last resort to gain management recognition of the company’s security risk. For the majority of us, such extreme measures are far too dangerous. In such cases, financial pressure from another source may be a last resort to gain management’s attention. Pressure from insurance companies (seeking to limit exposure of losses resulting from network-security incidents) will provide a financial incentive for organizations to improve security measures in order to keep insurance premiums affordable.

In a recent insurance application in which I was involved, an insurance company requested information on what policies an organization had in place for virus prevention and control of defamatory or libelous information on public Web sites and mailing lists. Conspicuous by its absence were questions seeking an understanding of how well prepared the organization was to prevent, detect, and respond to computer security incidents--even if only from the perspective of preventing viruses or defamatory/libelous information being published on a public forum. It will not be long before insurance companies are asking the right questions in this area. In fact some already are, but their motives are slightly different. Just recently some insurance companies have begun to offer policies that provide organizations with financial protection for third-party damages resulting from network-security breaches. A prerequisite for such coverage is an associated network-security risk assessment. It is only a matter of time before insurance companies begin to request more information about network security and begin to raise the cost of general insurance coverage for companies that are ill prepared to detect and respond to computer-security incidents. Eventually, one way or another, organizations through trial by fire or financial incentives will realize the need for a CSIRT.

Be prepared

It is still not uncommon to find callers to the CERT® Coordination Center hotline who do not know what steps to take to report an incident within their own organizations. Although many callers know who their vendor is and maybe even who the organization’s Internet service provider is, very few know to whom they should report a computer-security incident. Being prepared and knowing what to do in advance can help to further mitigate the damage from an incident. That is why it is very important that an organization advertise its CSIRT both internally and externally. As with emergency services, it is important to find out how to contact a CSIRT before it is needed in an emergency. It is also important to know in advance to whom the service can provide help and what information is needed to ensure that the CSIRT can provide the service requested.

To find out if your organization has a company-supported CSIRT, ask your security officer or system/network administrator, and consult your organization’s security policies and practices. Some CSIRTs are members of the Forum of Incident Response and Security Teams (FIRST). FIRST provides a list of its members and their contact information.

With millions of organizations now reliant on networks to conduct their businesses, it is a shocking fact that only a few hundred CSIRTs exist around the world today. Many of these CSIRTs continue to cite annual increases of 200% or 300% in the numbers of computer-security incidents reported to them and are struggling to keep pace with the number of incoming reports. Even with general improvements in the field of network security, a dramatic increase in the number of CSIRTs in existence today is urgently needed. More advocates are needed to help organizations understand the risks associated with the failure to detect and appropriately respond to computer-security incidents.

About the author

Moira J. West-Brown is a senior member of the technical staff within the CERT® Coordination Center, based at the SEI, where she leads a group responsible for facilitating and assisting the formation of new computer security incident response teams (CSIRTs) around the globe.

Before coming to the CERT®/CC in 1991, West-Brown had extensive experience in system administration, software development and user support/liaison, gained at a variety of companies ranging from academic institutions and industrial software consultancies to government-funded research programs. She is an active figure in the international CSIRT community and has developed a variety of tutorial and workshop materials focusing mainly on operational and collaborative CSIRT issues. She was elected to the Forum of Incident Response and Security Teams Steering Committee in 1995 and is currently the Steering Committee Chair. She holds a first-class bachelor's degree in computational science from the University of Hull, UK.

Find Us Here

Find us on Youtube  Find us on LinkedIn  Find us on twitter  Find us on Facebook

Share This Page

Share on Facebook  Send to your Twitter page  Save to del.ico.us  Save to LinkedIn  Digg this  Stumble this page.  Add to Technorati favorites  Save this page on your Google Home Page 

For more information

Contact Us



Help us improve

Visitor feedback helps us continually improve our site.

Please tell us what you
think with this short
(< 5 minute) survey.