Security Quality Requirements Engineering (SQUARE) Methodology
Nancy R. Mead
Eric D. Hough
Theodore R. Stehney II
Technical Report
CMU/SEI-2005-TR-009
November 2005
Networked Systems Survivability Program
[Abstract]
[Acknowledgements] [Executive
Summary] [1 Introduction]
[2 Security Requirements Engineering Process]
[3 Security Requirements Engineering Steps]
[4 Recent Results] [5
Future Research Plans] [Appendix]
[References] [PDF
File] The authors would like to acknowledge the work of the student teams at Carnegie
Mellon University that have contributed to the SQUARE methodology. They are
Peter Chen, Lydia Chung, Marjon Dean, Dan Gordon, Frank Hung, Lilian Lopez,
Don Ojoko-Adams, Hassan Osman, Will Salamon, Ted Stehney, Neha Wattas, Nick
(Ning) Xie, and Eugene Yu.
[Abstract]
[Acknowledgements] [Executive
Summary] [1 Introduction]
[2 Security Requirements Engineering Process]
[3 Security Requirements Engineering Steps] [4
Recent Results] [5 Future Research
Plans] [Appendix] [References]
[PDF
File]
The Software Engineering Institute's Networked Systems Survivability (NSS)
Program at Carnegie Mellon University has developed a methodology to help organizations
build security into the early stages of the production life cycle. The Security
Quality Requirements (SQUARE) Methodology consists of nine steps that generate
a final deliverable of categorized and prioritized security requirements. Although
the SQUARE Methodology could likely be generalized to any large-scale design
project, it was designed for use with information technology systems. The SQUARE process involves the interaction of a team of requirements engineers
and the stakeholders of an IT project. It begins with the requirements engineering
team and project stakeholders agreeing on technical definitions that serve as
a baseline for all future communication. Next, business and security goals are
outlined. Third, artifacts and documentation are created, which are necessary
for a full understanding of the relevant system. A structured risk assessment
determines the likelihood and impact of possible threats to the system. Following
this work, the requirements engineering team determines the best method for
eliciting initial security requirements from stakeholders, which is dependent
on several factors, including the stakeholders involved, the expertise of the
requirements engineering team, and the size and complexity of the project. Once
a method has been established, the participants rely on artifacts and risk assessment
results to elicit an initial set of security requirements. Two subsequent stages
are spent categorizing and prioritizing these requirements for management's
use in making tradeoff decisions. Finally, an inspection stage is included to
ensure the consistency and accuracy of the security requirements that have been
generated. SQUARE is a work in progress. Several case studies with real-world clients
have shown that the methodology holds good promise for incorporation into industry
practice. The SQUARE process has been enhanced and refined throughout the case
studies. The current working model is presented in this paper. NSS is currently
continuing research on the process and is working in parallel to create a CASE
tool to support each stage of the methodology.
[Abstract]
[Acknowledgements] [Executive
Summary] [1 Introduction] [2
Security Requirements Engineering Process]
[3 Security Requirements Engineering Steps]
[4 Recent Results] [5
Future Research Plans] [Appendix]
[References] [PDF
File]
Requirements engineering is well recognized in industry to be critical to
the success of any major development project. Several authoritative studies
have shown that requirements engineering defects cost 10 to 200 times more to
correct once fielded than they would if they were detected during requirements
development [Boehm 88, McConnell
01]. Other studies have shown that reworking requirements defects on most
software development projects costs 40 to 50 percent of total project effort
[Jones 86], and the percentage of defects originating
during requirements engineering is estimated at more than 50 percent [Wiegers
01]. A recent study found that the return on investment when security analysis
and secure engineering practices are introduced early in the development cycle
ranges from 12 to 21 percent, with the highest rate of return occurring when
the analysis is performed during application design [Soo
Hoo 01]. NIST reports that software that is faulty in security and reliability
costs the economy $59.5 billion annually in breakdowns and repairs [NIST
02]. The costs of poor security requirements show that there would be a
high value to even a small improvement in this area. By the time that an application
is fielded and in its operational environment, it is very difficult and expensive
to significantly improve its security. Requirements problems are the single number one reason projects Requirements engineering typically suffers from the following major problems: While much has been written about security requirements in the abstract, the
mechanisms to translate the theory into practice have been unclear. If security
requirements are not effectively defined, the resulting system cannot be effectively
evaluated for success or failure prior to implementation. Security requirements
are often missing in the requirements elicitation process. The lack of validated
methods is considered one of the factors. In addition to employing applicable software engineering techniques, the organization
must understand how to incorporate the techniques into its existing software
development processes. The identification of organizational mechanisms that
promote or inhibit the adoption of security requirements elicitation can be
an indicator of the security level of the resulting product. An earlier report, focusing on survivable requirements engineering, provided
an interesting range of material in this respect [Mead 03].
Security requirements engineering has only been recently researched, with much
of the material assembled within the past year or two. The security area did
not have techniques or templates for requirements elicitation. By assembling
an elicitation framework based on our initial research and applying it to active
software development efforts, we were able to identify additional research areas
and refine the framework through further research. If usable approaches to security are developed and mechanisms to promote organizational
use are identified, then the organization can ensure that the resulting product
effectively meets security requirements.
[Abstract]
[Acknowledgements] [Executive
Summary] [1 Introduction]
[2 Security Requirements Engineering Process]
[3 Security Requirements Engineering Steps]
[4 Recent Results] [5
Future Research Plans] [Appendix]
[References] [PDF
File]
Security Quality Requirements Engineering (SQUARE) has been developed at Carnegie
Mellon University by Nancy Mead with Donald Firesmith and Carol Woody of the
Software Engineering Institute (SEI). The process provides a means for eliciting,
categorizing, and prioritizing security requirements for information technology
systems and applications. The long-term goal of SQUARE is to integrate security
considerations into the early stages of the development life cycle. SQUARE has
also proven to be useful for documenting and analyzing the security aspects
of fielded systems and has potential for steering future improvements and modifications
to these systems. As is common with many quality requirements, the distinction between functional
and nonfunctional security requirements of a system is subtle. For instance,
a developer may argue that a system must protect against denial-of-service
attacks in order to fulfill its mission. Such a requirement may be considered
by other stakeholders to be an availability or reliability issue that is not
central to the mission of the system. In the SQUARE methodology, security requirements
are often discussed in the context of quality, non-functional requirements.
However, as opposed to artificially categorizing all security requirements
as nonfunctional, the methodology approaches security requirements as they are
often handled in requirements engineering: as an afterthought and add-on to
the system's functional requirements. By addressing security requirements in
this respect, the SQUARE Methodology is able to accurately adapt to the current
state of the practice of software and requirements engineering. The methodology is most effective and accurate when conducted with a team
of requirements engineers with security expertise and the stakeholders of the
project. The requirements engineering team can be thought of as external consultants,
though often the team is composed of one or more internal developers of the
project. Throughout this report, the terms "requirements engineer"
and "requirements engineering team" are synonymous. Likewise, this
report will refer to "stakeholders" also as "clients" or
"the client organization." The effectiveness of SQUARE in eliciting
requirements is dependent on representation from the project's stakeholders.
Thus, the requirements engineering team must emphasize the importance of establishing
a representative set of stakeholders to participate in the methodology. SQUARE can be decomposed into nine discrete steps, which are outlined in Table
1. Each step identifies the necessary inputs, major participants, suggested
techniques, and final output. In Section 3
of this report, the steps are explained in detail. Generally, the output of
each step serves as the sequential input to the following steps, though some
steps may be performed in parallel. For instance, it might be more efficient
for the requirements engineering team to perform Step 2 (Identify Security Goals)
and Step 3 (Develop Artifacts) simultaneously, since to some extent they are
independent activities. The output of both steps, however, is required for Step
4 (Identify Security Goals). Table 1: Steps in the SQUARE Process As Table 1 illustrates, the first task
for the organization is to agree on a common set of security definitions, followed
by the definition of organizational security goals. These security goals may
be derived from business application goals, potential threats to project assets,
and management controls and policy. The requirements engineer can then develop
artifacts (network maps and diagrams, attack trees, and use/misuse cases) that
will aid in the elicitation of security requirements. Next, a formal risk assessment
allows the organization to understand how the likelihood and impact of various
threats can affect the project's security goals and assets. At this point in the SQUARE process, the requirements engineer must select
one or more requirements-elicitation techniques appropriate for the organization's
culture, expertise, level of security required, and nature of the system being
developed. The selected techniques are subsequently used to elicit an initial
set of security requirements in the form of operational constraints on the system.
An example of such a requirement is "The system shall only reveal employee
salary information to members of the Human Resources Department." These
requirements are then categorized, prioritized, and formally inspected for correctness
in the remaining steps of SQUARE. The final output of the process is a security
requirements document that satisfies the security goals of the project, contains
clear and verifiable requirements, and is agreed on by all relevant stakeholders.
[Abstract]
[Acknowledgements] [Executive
Summary] [1 Introduction]
[2 Security Requirements Engineering Process]
[3 Security Requirements Engineering Steps] [4
Recent Results] [5 Future Research
Plans] [Appendix]
[References] [PDF
File]
In this section, the steps in the SQUARE Methodology are enumerated in greater
detail. The purpose of this section is to clarify the purpose of each step,
the expectations of the participants, and the exit criteria of each step. Recommendations
to the process are also stated where appropriate. In order to guarantee effective and clear communication throughout the requirements
engineering process, the requirements engineering team and stakeholders must
first agree on a common set of terminology and definitions. Given the differences
in expertise, knowledge, and experience, an arbitrary term may have multiple
meanings between the participants of SQUARE. In addition, there may be ambiguity
in the level of detail that is assumed for a given term. For instance, one stakeholder
may view "access controls" as a set of policies that governs which
users may be granted access to which resources. Another stakeholder may view
access controls as the software elements in the system that actually implement
this functionality. These differences in perspective must be resolved before
the process can continue. Fortunately, it is not necessary to reproduce a comprehensive set of definitions
for each iteration of SQUARE. Using public resources, such as the Software Engineering
Body of Knowledge (SWEBOK) [IEEE 05], IEEE 610.12 Standard Glossary of Software
Engineering Terminology [IEEE 90], and Wikipedia [Wiki
05], the requirements engineering team can produce and reuse a comprehensive
set of security terms with which to work. See Table
2 for an example set of initial terms upon which the requirements engineering
team can build. Table 2: Example Set of Initial Terms The initial list of terms should also include suggested definitions for each
term and their corresponding sources. This allows the stakeholders to get a
general understanding and scope of each term, and in the common case, select
one of the suggested definitions as final. See Table
3 for an example of the information that should be provided with each term.
In this example, the stakeholders could place a checkmark next to the definition
that suits them best. Table 3: Example Term with Suggested Definitions The property that
information is not made available or disclosed to unauthorized individuals,
entities, or processes. (i.e., to any unauthorized system entity) [SANS
03a] Ensuring that information is
available to only those with authorized access. [ISO
04] Restricting access to information
via a hierarchy of classes of access. [JONES
02] Other: _______________________________________________________
Select an existing or create a custom definition for each term provided by
the requirements engineering team. All stakeholders must come to a consensus
on each term's definition in a timely manner and present their results to the
requirements engineering team. Establish a single point of contact (POC) for interaction between the requirements
engineering team and the stakeholders. A well-documented, agreed-on set of definitions has been established and is
available to all stakeholders and the requirements engineering team. The definitions
document will be used as a reference throughout the rest of the SQUARE process.
The purpose of Step 2 in SQUARE is for the stakeholders to formally agree
on a set of prioritized security goals for the project. Without overall security
goals for the project, it is impossible to identify the priority and relevance
of any security requirements that are generated. In addition, the establishment
of security goals scopes the rest of the SQUARE process. Initially, different stakeholders will likely have different security goals.
For example, a member of human resources may be concerned about maintaining
the confidentiality of personnel records, whereas a stakeholder in finance may
be concerned with ensuring that financial data is not modified without authorization.
The security goals of the stakeholders may also conflict with one another. A
security-conscious stakeholder may place high importance on strong security
controls for the system, which in turn may hamper overall system performance.
Decreased performance might likely be at odds with the goals of the marketing
department. Step 2 in the SQUARE process serves to eliminate such conflicts
and align all of the stakeholders' interests. The security goals of the project must be in clear support of the project's
overall business goal, which also must be identified and enumerated in this
step. On average, stakeholders should attempt to brainstorm to come up with
approximately half a dozen security goals for the project, with more or less
depending on the scale of the project. More sophisticated techniques for mapping
high-level business requirements to low-level requirements can be found in Core
Security Requirements Artefacts [Moffett 04] and
"Mapping Mission-Level Availability Requirements to System Architectures
and Policy Abstractions" [Watro 01]. Once the goals of the various stakeholders have been identified, they must
be prioritized. In the absence of consensus, an executive decision may be needed
to prioritize the goals. Finally, the requirements engineering team must encourage the stakeholders
to generate security goals as opposed to requirements or recommendations. There
is a fine line between a security goal such as "The system shall
be available for use when needed," a requirement such as "The
system must have a continuity of operations plan in place to ensure appropriate
system availability," and a recommendation such as "Invest
in backup information technology hardware to ensure business continuity."
The requirements engineering team must act as the experts in this situation,
providing assistance to the stakeholders so that they may generate an appropriately
scoped set of security requirements. A single business goal for the project and several prioritized security goals
that support it have been established. Before the requirements engineering team and stakeholders can generate a comprehensive
set of security requirements, the team must collect a complete set of artifacts
of the system. The following are the types of artifacts that should be collected: In developing such artifacts, it is important to enlist the assistance of
knowledgeable engineers from the organization. In some cases, it is possible
that the client organization will not have any of the artifacts in place,
including such basic items as system architecture diagrams. In such situations,
the requirements engineering team should reiterate to the stakeholders that
by creating and documenting artifacts of the system, they are investing in the
success of the project. Work with the stakeholders and client organization to identify and collect
as many artifacts as possible. Generate or collect any system artifacts and present them to the requirements
engineering team. This information includes system architectures, actors, use/misuse
cases, and suspected attacks. It is possible that other experts from within
the organization may be called upon to provide information as well. Verify the accuracy and completeness of all artifacts. A set of artifacts for the system, as complete as possible, has been generated
by the requirements engineering team and shared with the stakeholders. The purpose of this step in the SQUARE process is to identify the vulnerabilities
and threats that face the system, the likelihood that the threats will materialize
as real attacks, and any potential consequences of an attack. Without a risk
assessment, organizations can be tempted to implement security requirements
or countermeasures without a logical rationale. For instance, the stakeholders
may decide that encryption is a necessary component of their system without
fully understanding the nature of the problem that encryption can solve. The
risk assessment also serves to prioritize the security requirements at a later
stage in the process. There are a growing number of risk assessment methods from which to choose
(see the list of examples in Section 4.5.1).
Some of the methods are very structured and may require the assistance of an
external risk expert. Ideally, this expert would already be a part of the requirements
engineering team. After the threats have been identified by the risk assessment method, they
must be classified according to likelihoods. Again, this will aid in prioritizing
the security requirements that are generated at a later stage. For each threat
identified, a corresponding security requirement can identify a quantifiable,
verifiable response. For instance, a requirement may describe speed of containment,
cost of recovery, or limit to the damage that can be done to the system's functionality.
All vulnerabilities and threats have been identified and classified according
to their likelihoods. Potential consequences of attacks are identified. The
results are well documented and shared with the stakeholders. The requirements engineering team must select an elicitation technique that
is suitable for the client organization and project. Although this task may
appear to be straightforward, it is often the case that multiple techniques
will likely work for the same project. The difficulty is in choosing a technique
that can adapt to the number and expertise of stakeholders, size and scope of
the client project, and expertise of the requirements engineering team. It is
extremely unlikely that any single technique will work for all projects under
all circumstances, though previous experience has shown that the Accelerated
Requirements Method (ARM) has been successful in eliciting security requirements.
This particular technique is discussed further in Section
4.7.2. The following is a sample of elicitation techniques that may be appropriate: The requirements engineering team has selected an appropriate elicitation
technique and documented the rationale for their choice. This step is the heart of the SQUARE process: the elicitation of security
requirements. To the benefit of the requirements engineering team, most elicitation
techniques provide detailed guidance on how to perform the elicitation, so this
step is simply a matter of executing the technique. However, even if the stakeholders
are very knowledgeable about the project and communicate effectively, it can
be challenging for the requirements engineering team to elicit correct requirements.
Perhaps the largest mistake that the requirements engineering team can make
in this step is to elicit non-verifiable or vague, ambiguous requirements. Each
requirement must be stated in a manner that will allow relatively easy verification
once the project has been implemented. For instance, the requirement "The
system shall improve the availability of the existing customer service center"
is impossible to measure objectively. Instead, the requirements engineering
team should encourage the production of requirements that are clearly verifiable
and, where appropriate, quantifiable. A better version of the previously stated
requirement would thus be "The system shall handle at least 300 simultaneous
connections to the customer service center." A second mistake that the requirements engineering team can make in this step
is to elicit implementations or architectural constraints instead
of requirements. Requirements are concerned with what the system should
do, not how it should be done. All elicitation techniques will involve face-to-face interaction with the
stakeholders, so it is also the responsibility of the requirements engineering
team to make logistical arrangements with the stakeholders and inform them of
the time they can expect to spend in this part of the SQUARE process. Follow the instructions given by the requirements engineering team during
the elicitation process. Encourage the generation of verifiable, preferably quantifiable security requirements.
An initial set of security requirements for the system has been elicited and
documented. It is not necessary that the set be considered final or completely
correct. The purpose of this step is to allow the requirements engineer and stakeholders
to classify the requirements as essential, non-essential, system level, software
level, or as architectural constraints. The requirements engineering team can
provide to the stakeholders a matrix such as the one in Table
4 to assist in this process. Table 4: Minimal Set of Categories for Requirements Categorization System level Software level Architectural constraint
Essential Non-essential The categories in Table 4 are not fixed;
each iteration of SQUARE will likely produce a much larger set of categories
that are customized to the project at hand. These categories are instead suggested
as a minimal set. Since the goal of SQUARE is to produce security requirements, the requirements
engineering team and stakeholders should avoid producing architectural constraints.
Architectural constraints are provided as a category here to serve as an outlet
for "requirements" that, upon categorization, are considered to be
constraints. Ideally, such anomalies would be identified and corrected in the
previous steps of the process. Once the requirements are categorized, the requirements engineering team and
stakeholders will be able to prioritize them more efficiently. Come to a consensus on the categorization for each requirement. The initial set of requirements has been organized into stakeholder-defined
categories, and any remaining architectural constraints are identified as such.
In most cases, the client organization will be unable to implement all
of the security requirements due to lack of time, resources, or developing changes
in the goals of the project. Thus, the purpose of this step in the SQUARE process
is to prioritize the security requirements so that the stakeholders can choose
which requirements to implement and in what order. The results of Step 4, the
risk assessment, and Step 7, categorization, are crucial inputs to this step.
The available prioritization methods are flexible and can be as simple as
unstructured deliberation between the stakeholders. There are several structured
prioritization techniques that exist, such as Triage [Davis
03], Win-Win [Boehm 01], and the Analytical Hierarchy
Process (AHP); the latter has been reported to be quite effective [Karlsson
97, Saaty 80]. AHP is discussed in detail in Section
4.9 of this report. Ideally, the requirements engineering team should also
produce a cost-benefit analysis to aid the stakeholders' decisions. During prioritization, some of the requirements may be deemed to be entirely
unfeasible to implement. In such cases, the requirements engineering team has
a choice: completely dismiss the requirement from further consideration, or
document the requirement as "future work" and remove it from the working
set of project requirements. This decision should be made after consulting with
the stakeholders. Facilitate the prioritization process with the stakeholders. If a structured
prioritization process is selected, teach the stakeholders how to perform the
process. Prioritize the security requirements using the risk assessment and categorization
results as a basis for decision making. All security requirements have been prioritized. The last step of the SQUARE process, requirements inspection, is one of the
most important elements in creating a set of accurate and verifiable security
requirements. Inspection can be done at varying levels of formality, from Fagan
Inspections to peer reviews [Fagan 86, Wiegers
02]. The goal of any inspection method, however, is to find any defects
in the requirements such as ambiguities, inconsistencies, or mistaken assumptions.
Facilitate the inspection process by providing any orientation to the structured
inspection technique or informal inspection guides such as checklists. Come to a consensus on the validity of each security requirement. Verify that
each requirement is verifiable, in scope, within financial means, and feasible
to implement. Requirements that do not fit these criteria should have been identified
in earlier stages of SQUARE, but the stakeholders should use this opportunity
as a last chance to remove any requirements from the working set. Verify that each requirement is directly applicable to one or more of the
security goals of the project or in support of a higher level requirement. All security requirements have been verified both by the requirements engineering
team and the stakeholders. At this point the SQUARE process is complete, and
the requirements engineering team can produce the final security requirements
document for the stakeholders.
[Abstract]
[Acknowledgements] [Executive
Summary] [1 Introduction]
[2 Security Requirements Engineering Process]
[3 Security Requirements Engineering Steps]
[4 Recent Results] [5 Future
Research Plans] [Appendix]
[References] [PDF
File]
The SQUARE Methodology has undergone several case studies conducted by graduate
students at Carnegie Mellon University [Chen 04, Gordon
05]. The goals of the case studies were to experiment with each step of
the SQUARE process, make recommendations, and determine the feasibility of integrating
SQUARE into standardized software development practices. The case studies involved
real-world clients that were developing large-scale IT projects. The clients
included an IT firm in Pittsburgh, Pennsylvania, a federal government research
institute, and a department of the federal government. In this section, the
results from the case studies are presented at each step of the process. The process of establishing and agreeing on definitions was conducted rather
straightfors academic courses and prior work experience. outlines some of these
terms, and Section lists the final set of terms along with their agreed-on definitions.
After narrowing the potential definitions for each term down to two or three,
along with a "suggested" definition, the team sought input from the
stakeholders by emailing a set of compiled definitions with appropriate referencing.
The team asked the client to indicate which definitions best fit their understanding,
to modify or create new definitions, and to add any terms that may have been
omitted. The following figure shows a snapshot of part of the document that
was submitted: Figure 1: Sample of Terms and Definitions Provided to Stakeholders for Review
The clients relied on the requirements engineering team's expertise and knowledge
for guidance through the first step; stakeholders considered some terms to overlap
and have ambiguous or double meanings. For a large portion of the terms, the
suggested definition was selected by the stakeholders. After each term had a
finalized definition, the requirements engineering team finalized the dictionary
of terms and shared it with the stakeholders. The identification of security goals for each of the case studies was performed
with varied difficulty. Some of the case studies had identified business and
security goals before the SQUARE process even began. Others did not even have
an explicit business goal for their system. In this section of the report, the
development of Acme Corporation's security goals is presented as an example.
To begin with, the student team asked the stakeholders of Acme to develop
a business goal that could be expressed in a single sentence. In this case,
the stakeholders came up with "The system allows the client to make informed
decisions based on which assets are available." The team then asked the
stakeholders to generate a few security goals for the product that were in more
or less clear support of this business goal. They created a simple hierarchy,
shown in , that illustrated how the system's business goal, security goals,
security requirements, and recommendations connected together. In this case,
Acme developed one business goal and three security goals, which are shown in
Table 5. The corresponding security requirements,
which were generated in a later stage of the process, are presented in Section
4.7. Figure 2: Simple Hierarchy of Goals and Recommendations Table 5: Acme Corporation's Business and Security Goals
Throughout the case studies, a series of artifacts were developed that served
as important inputs to subsequent steps. The artifacts included system architecture
diagrams, use/misuse cases, attack trees, and assessment of essential assets
and services. The case studies revealed that some organizations, surprisingly, had no documentation
of the system architecture for their projects. In some cases, this was due to
the dynamic nature of the project: there was no "typical" setup of
the system due to the fact that the end user more or less defines that. Regardless,
the establishment of system architecture diagrams proved very useful throughout
the later stages of SQUARE. It is possible that the requirements engineering
team will need to handle very detailed system architecture diagrams. The following
figure is an example of a simple system architecture diagram that was produced
by one of the case studies. Figure 3: Example System Architecture Diagram <,/p>
Attack trees provide a formal, hierarchical way of describing the security
threats to a system based on the types of attacks that could happen and how
they could be realized. Attack tree diagrams represent attacks in a tree structure,
where an attacker's goal is listed as the root node and tree leaves represent
different ways to achieve that goal. In the following example, a higher level
attack tree is depicted, followed by a drill down of a more specific sub tree:
Figure 4: Example Attack Tree In general, attack trees proved very useful to both the stakeholders and the
requirements engineering team in getting a better sense of the scope of threats
that the system faces. Many attack trees can be reused between iterations of
SQUARE, thus saving the requirements engineering team from developing entirely
new attack trees for the same attacks. Use cases are scenario-based artifacts that force stakeholders to answer "how
is this done" questions from a user's perspective. By providing a context
for operation, stakeholders and the requirements engineering team can gain a
deep understanding of the interactions of the system components. Use case scenarios
can be supplemented with use case diagrams, which graphically display the component
interactions of the system. A detailed set of use cases were completed during the SQUARE case studies,
which both the requirements engineering team and stakeholders found to be very
useful. A sample use case and diagram are shown in Table 6
and Figure 5: Figure 5: Sample Use Case Diagram For the initial case study, a detailed set of misuse cases were completed
that encompassed the most significant threats to the system. Sample misuse case
and diagram traces are shown in Table 7
and Figure 6. Table 7: Example Misuse Case Figure 6: Example Misuse Diagram Trace One of the goals of completing the attack trees was to ensure that the team
had a complete set of misuse cases. shows a mapping between the attack trees
and the misuse cases. While the attack trees provide a general picture of potential
attacks on the system, the misuse cases drill down to the details of the interactions
between system components in the event of an attack. Table 8: Mapping Between Misuse Cases and Attack Trees Mapping the attack trees to the misuse cases provided a useful sanity check
for the work. Additionally, the team found that it might have been useful to
have two independent teams working in parallel: one team working on attack trees
and one team working on misuse cases. In one of the case studies, the team noticed that another class of artifacts
could be derived by utilizing the Survivable Systems Analysis (SSA) method developed
by the CERT Coordination Center (CERT/CC) [CERT/CC 02].
SSA is a white-team exercise aimed at providing survivable recommendations for
a system. Some of the preliminary work for SSA is already covered in the earlier
stages of SQUARE. The team noticed that Step 2 of SSA, defining essential service
scenarios and components, would yield useful artifacts for inclusion in the
case study. To begin identifying the essential elements of the system, the team first
looked back to the business goal of the client's project. In this case, the
goal was to "provide the ability to make important decisions in emergency
situations based upon current and available information." The students
analyzed the use cases of the system and made a determination as to which services,
assets, and components were essential to fulfilling the business goal of Acme's
product. The team analyzed the importance of each of the major system services, outlined
in Table 9 by way of use cases, and made
a determination as to each service's essentiality. Table 9: Use Cases and Initial Rankings of Essentiality The major business goal of this particular system was to allow decisions to
be made both before an emergency takes place (i.e., in the planning phase),
as well as during and after an event. The most critical services needed to assist
decision making are those that directly affect viewing and altering event-specific
information. Thus, viewing floor plans, entering damage assessments, creating
open space reports, and viewing incident commands would be of highest priority.
If an emergency or an attack were to occur, it would be crucial to preserve
these system functions. Though not quite critical, the case study team flagged
two services as important: viewing contact information for maintenance tasks,
and finding specialized employees. The other major functions were all deemed to be non-essential. Though important
to the functioning and upkeep of the system, the ability to add this information
can be recovered after an attack. Many of the other functions deal with configuring
the actual system or its user profiles, which can also be performed after an
attack with little loss. Other functions involve making useful but non-critical
posts in the form of journal entries or Post-It notes. Still other services
support the viewing of non-critical data such as overseas contact information.
While all of this functionality is important to the long-term usability of the
system, an attack on these services does not threaten the ability of the system
to aid decision making during an emergency. If compromised, the information
and services would need to be repaired before the system could become fully
usable and functional again. However, if the information in the system configuration
is kept current, the ability to add new assets and documents during an emergency
is secondary to viewing the current state of assets. There were two major assets in the system under study: (1) a server that housed
the majority of the system's intellectual assets (i.e., the code that ran the
system) and provided remote access and (2) the information inside the central
server, including Microsoft IIS configurations, the Sybase database, and the
MapGuide Database. These assets were found to be critical in order to make informed
decisions. The user and developer workstations in the system were not considered. No
important files or intellectual assets critical to system's mission are housed
on these machines. Should they fail, a spare machine could easily act as a replacement,
provided the proper software is available. This is not the case with the central
server or the information that it contains. An attack on its ability to function,
or on its ability to deliver accurate information, will critically impact survivability.
In the case study analyses, some risk assessment methodologies were analyzed
to determine which were suitable for the elicitation of security requirements.
The result of this analysis is presented in this section as an example of techniques
that are available, as well as their strengths and weaknesses. These techniques
are provided only as examples of previous work, not as recommendations for inclusion
in the SQUARE process. Before the first SQUARE case study, the student team performed a literature
review of the available risk assessment techniques. Ideas came from faculty,
course work completed by team members at Carnegie Mellon, and Internet and library
searches. The search was narrowed down to a list of eight techniques: After the initial research, the team completed a brief analysis to determine
which models would be likely candidates for further consideration. They found
that attempts to quantify risks on the basis of dollar value per attack were
either too complicated or too involved for the limited time given to the project,
and were therefore rejected. The team concluded that qualitative methods would
add more value to the short case studies. The following chart shows which criteria were used to evaluate the different
methodologies and how each was scored (using a scale of 1 - 4, with "1"
being the highest mark, and "4" being the lowest). Here is a brief
explanation of each rating: Table 10: Results of Risk Assessment Literature Review As Table 10 illustrates, the student
team found that NIST's SP 800-30, also called the "Risk Management Guide
for Information Technology Systems," and Yacov Haimes's "Risk Filtering,
Ranking, and Management Framework" (RFRM) held the most promise for inclusion
in the SQUARE process. The student teams proceeded to conduct an independent field test for each
of the two selected methodologies. This section outlines the results of their
work. The RFRM framework contains eight phases, some of which the team found to
be out of scope for a basic risk assessment. The team identified and tested
two phases of RFRM that it felt were strong candidates for inclusion in SQUARE:
Phase III, Bicriteria Filtering and Ranking, and Phase IV, Multicriteria Filtering
and Ranking. NIST's model for risk assessment is broken into nine steps, each with an output
that serves as the input to the next step. Steps 1, 8, and 9 were omitted from
the test; Step 1 was completed previously in the SQUARE process, Step 8 deals
with control recommendations, which are handled separately in SQUARE, and Step
9, Documentation, was omitted because the team combined these results with the
RFRM results. Thus, the steps that were actually included were The results of the different approaches in these two models produced a list
of security risks that are on different levels of abstraction, and thus the
two sets of filtered and ranked risk scenarios cannot always be easily compared.
In some cases, the results of the two models were in conflict. In other cases,
the models produced similar evaluations of risk scenarios. The following chart
summarizes a three-tier view of each model's risk assessment results on one
of the clients' systems: Table 11: Risk Assessment Results The team analyzed the combined results and was able to make the following conclusions: Both risk assessment models are concerned with hardware failure or destruction,
but they rank the importance differently. Hardware damage is a "Tier 2"
risk for both models, but NIST's output considers deliberate destruction by
an insider or terrorist a "Tier 1" risk. Some of the risk scenarios
from each model do not map directly to one another. NIST's output focuses more
on an attacker's motives once inside the system (destroying and corrupting data,
disclosing proprietary information, etc.) whereas RFRM's output deals more with
the ability of an attacker to break the frontline defenses of the system. Every application of the SQUARE Methodology will be unique, and so too will
the risk assessment, as it needs to be tailored to meet the context of the system
under analysis. What is important is that the results from the risk assessment
provide a meaningful way to categorize the likelihood and impact of the major
threats to the system. In order to investigate the applicability of existing requirements elicitation
techniques to security requirements, one of the case study teams performed a
literature review of the existing techniques. Their results are presented in
this section. The team researched existing, structured elicitation techniques and evaluated
them on the following criteria: Once these criteria were defined, the student team
produced the comparison matrix shown in Table
12. The techniques they investigated were misuse cases [Jacobson
92], Soft Systems Methodology (SSM) [Checkland 89],
Quality Function Deployment (QFD) [QFD 05], Controlled
Requirements Expression (CORE) [Mullery 79], Issue
Based Information Systems (IBIS) [Kunz 70], Joint Application
Development (JAD) [Wood 89], Feature-Oriented Domain Analysis
(FODA) [Kang 90], Critical Discourse Analysis (CDA) [Schiffrin
94], and the Accelerated Requirements Method (ARM) [Hubbard
99].
Table 12: Comparison of Elicitation Techniques Scale: 3 = very good, 2 = fair, 1 = poor. Based on their comparison, the student team decided to pursue IBIS, JAD, and
ARM for further consideration. The result of their experience with each technique
is presented in the following section. The very first case study team did not use a structured elicitation technique
to develop requirements with the stakeholders. In essence, this team utilized
the "unstructured interview" method of requirements elicitation. Working
with their client, Acme Corporation, they were able to generate the set of requirements
shown in Table 13. Table 13: Requirements Generated for Acme Using the "Unstructured
Interview" Elicitation Technique
Acknowledgments
Executive Summary
1 Introduction
1.1 Industry Problem
1.2 Practice Description
2 Security Requirements Engineering Process
3 Security Requirements Engineering Steps
3.1 Step 1: Agree on Definitions
Requirements Engineering Team Responsibilities:
Stakeholder Responsibilities:
Joint Responsibilities:
Exit Criteria:
3.2 Step 2: Identify Security Goals
Requirements Engineering Team Responsibilities:
Stakeholder Responsibilities:
Exit Criteria:
3.3 Step 3: Develop Artifacts
Requirements Engineering Team Responsibilities:
Stakeholder Responsibilities:
Joint Responsibilities:
Exit Criteria:
3.4 Step 4: Perform Risk Assessment
Requirements Engineering Team Responsibilities:
Exit Criteria:
3.5 Step 5: Select Elicitation Technique
Requirements Engineering Team Responsibilities:
Exit Criteria:
3.6 Step 6: Elicit Security Requirements
Requirements Engineering Team Responsibilities:
Stakeholder Responsibilities:
Joint Responsibilities:
Exit Criteria:
3.7 Step 7: Categorize Requirements
Requirements Engineering Team Responsibilities:
Stakeholder Responsibilities:
Exit Criteria:
3.8 Step 8: Prioritize Requirements
Requirements Engineering Team Responsibilities:
Stakeholder Responsibilities:
Exit Criteria:
3.9 Step 9: Requirements Inspection
Requirements Engineering Team Responsibilities:
Stakeholder Responsibilities:
Joint Responsibilities:
Exit Criteria:
4 Recent Results
4.1 Step 1: Agree on Definitions
4.2 Step 2: Identify Security Goals
4.3 Step 3: Develop Artifacts
4.3.1 System Architecture
4.3.2 Attack Trees
4.3.3 Use Cases
4.3.4 Misuse Cases
4.3.5 Reconciliation of Attack Trees and Misuse
Cases
4.4 Essential Assets and Services
4.4.1 Essential Services
4.4.2 Essential Assets
4.5 Step 4: Perform Risk Assessment
4.5.1 Risk Assessment Techniques
4.5.2 Risk Assessment Field Tests
4.6 Step 5: Select Elicitation Techniques
4.6.1 Literature Review
The ability of the technique to produce accurate requirements in diverse environments. For
example, does the technique apply only to functional requirements?
Does the technique have a software tool to complement the process?
The likelihood that the client would agree to the elicitation technique in
analyzing their requirements. Is the process too invasive in a business environment?
The degree of difficulty in understanding and properly executing the elicitation
technique. Can the requirements engineers and stakeholders easily perform
the technique correctly once they learn the process?
The ability of the elicitation technique to produce readily understandable
visual artifacts that appeal to the stakeholders.
The length of time the requirements engineers and clients need to fully execute
the elicitation technique.
The speed with which the requirements engineers and clients can fully comprehend
the elicitation technique.
The time, exposure, and analysis the elicitation technique has experienced
in its vetting by the requirements engineering community.
The ability of the elicitation technique to address the requirements of enterprise-level
systems, in addition to smaller applications. 4.7 Step 6: Elicit Security Requirements