Using the SEI Architecture Tradeoff Analysis Method to Evaluate WIN-T: A Case Study
Paul Clements
John Bergey
Dave Mason
Technical Note
CMU/SEI-2005-TN-027
Software Architecture Technology Initiative
Unlimited distribution subject to the copyright.
[3 The ATAM]
[6 Conclusion]
Acknowledgements
We thank Lawrence Jones and Liam O'Brien of the Carnegie Mellon Software Engineering Institute for their thorough review of this technical note.
[3 The ATAM]
[6 Conclusion]
1 Introduction
Because software architecture is a major determinant of software quality, it follows that software architecture is critical to the quality of any software-intensive system [Clements 96]. For a Department of Defense (DoD) acquisition organization, the ability to evaluate software architectures before they are realized in finished systems can substantially reduce the risk that the delivered systems will not meet their quality goals.
To meet this need for architecture evaluation, the Carnegie Mellon Software Engineering Institute (SEI) developed the Architectural Tradeoff Analysis Method (ATAM) and validated its usefulness in practice [Kazman 00]. This method not only permits evaluation of specific architecture quality attributes (e.g., modifiability, performance, security, and reliability) but also allows engineering tradeoffs to be made among possibly conflicting quality goals.
This technical note describes an ATAM evaluation of the software architecture of the Warfighter Information Network-Tactical (WIN-T) system, which is a sophisticated communications network. This system is being developed by a government-contractor team led by the U.S. Army's Communications and Electronics Command (CECOM) at Fort Monmouth, New Jersey.
Following this introduction, Section 2 defines software architecture, explains the importance of architecture evaluation, and provides an overview of the WIN-T system. Section 3 contains an overview of the ATAM including its purpose and primary steps. Section 4 describes how the ATAM was applied specifically to WIN-T and the results of that application. Section 5 describes some of the post-ATAM activities that resulted from the evaluation, and Section 6 summarizes the overall evaluation.
[3 The ATAM]
[6 Conclusion]
2 Context for the Architecture Evaluation
2.1 Software Architecture
The software architecture of a program or computing system is the structure or structures of the system, which comprise software elements, the externally visible properties of those elements, and the relationships among them
[Bass 03].
The software architecture of a system embodies the earliest software design decisions. These decisions enable or preclude achievement of desired system qualities, such as reliability, modifiability, security, real-time performance, and interoperability. The architecture also forms the basis for the development approach and acts as the blueprint that guides the teams building the system. Architectural decisions are the most critical to get right and the most difficult to change downstream in the development life cycle. The right software architecture paves the way for successful system development. The wrong architecture will result in a system that fails to meet critical requirements, suffers from budget and schedule overruns, and incurs high maintenance costs.
Modern approaches to software architecture take a multi-view approach. A view is a representation of one or more structures present in the system. If we consider the analogy of the architecture of a building, various stakeholders (such as the construction engineer, the plumber, and the electrician) all have an interest in how the building is to be constructed. Although they are interested in different components and different relationships, each of their views is valid. Each view represents a structure that maps to one of the construction goals of the building, and these multiple views are necessary to represent the architecture of the building fully. Similarly, a software architecture has a variety of stakeholders, including developers, maintainers, testers, integrators, system administrators, project managers, analysts, certification authorities, end users, and the acquisition organization. Each of these stakeholders has a vested interest in different system properties and goals that are represented by different structural views of the system. The views provide the basis for reasoning about the appropriateness and quality of the architecture for achieving system quality goals.
Some common architectural views include [Clements 02a]
- the logical view, which represents system functions; key system abstractions and their dependencies; and data flows
- the module decomposition view, which represents the hierarchical decomposition of the system's functionality into units of implementation. This decomposition can include objects, procedures, functions, and their relationships.
- the communicating-processes view, which represents processing threads, their synchronization, and the data flows between them
- the deployment view, which shows how the software is allocated to hardware including processors, storage, and external devices or sensors, along with the communications paths that connect them
Other important software architectural views are described in Documenting Software Architecture: Views and Beyond [Clements 02a]. Views serve as the primary vehicle for communicating the architecture to its stakeholders, the primary engineering handle for designing quality attributes into the system, and the primary window into the architecture for evaluators who are checking it for fitness of purpose.
Because the architecture is important in system development, it is also important in system acquisition. It is the responsibility of the acquisition organization to ensure that the architecture will support attainment of system quality goals. Formal architecture evaluation is an essential part of an architecture-based acquisition effort, as is insuring that high-quality architecture documentation is produced and maintained.
2.2 The WIN-T Organization
The acquisition organization for WIN-T is CECOM at Fort Monmouth, New Jersey. The Program Executive Office (PEO) Command, Control, and Communications Tactical (C3T), which is collocated at Fort Monmouth, is the management entity responsible for WIN-T. Under the PEO, the day-to-day execution is managed by the program manager of WIN-T (PM, WIN-T), who, in turn, is supported by a project director (PD) and project team. That team is augmented and assisted by various support divisions of the CECOM and PM, WIN-T organization. Figure 1 depicts the WIN-T organizational infrastructure at the time the ATAM-based architecture evaluation was held. Since then, CECOM, PEO C3T, and PEO Intelligence, Electronic Warfare, and Sensors (IEWS) and their subordinate elements have merged into the Communications Electronics Life Cycle Management Center (CE LCMC).
Figure 1: WIN-T Organizational Infrastructure
Three organizations within CECOM provide matrix support to the PM, WIN-T: (1) the Acquisition Center, (2) the Battle Space Systems Division of the Software Engineering Center (SEC), and (3) the Logistics Readiness Command (LRC). Additional external support is provided by the Training and Doctrine Command (TRADOC) Systems Manager (TSM) who is assigned to WIN-T (TSM, WIN-T). The TSM, WIN-T resides at the Fort Gordon Signal Center in Georgia and is the end-user representative responsible for the development of the Operational Requirements Document (ORD).
Within CECOM, the Acquisition Center supports a large number of CECOM organizations including the SEC and a number of PEOs, including PEO C3T. The Acquisition Center, which is home to the contracting officer (KO), provides a full spectrum of acquisition services. Major commodities include aviation communications, man-portable radios, radar systems, computers, satellite communications, night-vision equipment, command-and-control systems, sensors, information management systems, battery and power sources, intelligence/electronic warfare systems, mines/countermines, facilities supplies, and a host of technical services that support the various mission responsibilities of the center's customers.
The SEC provides life-cycle software products and services from the battle space through the sustaining base. The Battle Space Systems Division is the organizational element within the SEC that is responsible for providing centralized software life-cycle engineering, management, and support for more than 150 mission-critical defense systems, including embedded matrix software support to the PM, WIN-T.
2.3 The WIN-T System
WIN-T is the Army's tactical telecommunications system consisting of communication infrastructure and network components designed to provide secure communications at all echelons from the Maneuver Battalion to the Theater Rear boundary. WIN-T is required to be the high-speed, high-capacity, high-mobility backbone communications network, providing voice, data, and video service, for what is referred to as the Army's "Future Force." WIN-T is to set standards and protocols for the Future Force while interfacing with and/or replacing equipment in the Army's current forces and interim Stryker forces. These forces include the Mobil Subscriber Equipment (MSE) that provides communications from Corps to Brigade and the Tri-Service Tactical Communications System (TRI-TAC) that provides communications at Echelons Above Corps (EAC). Moreover, WIN-T will provide command centers and staff elements at the Unit of Employment (UE) with the communications capabilities to link to adjacent UEs, subordinate Maneuver Brigades/Units of Action (MBs/UAs), and sustaining base, Joint, Allied, and Coalition forces. WIN-T is to provide required reach,1 reachback,2 and network operations for the MBs/UAs and interface seamlessly with the Joint Tactical Radio System (JTRS) that extends to the individual warfighter platform.
The operational concept graphic shown in Figure 2 depicts how the WIN-T network interconnects UE elements (at Division, Corps, and Theater echelons), as well as the WIN-T presence in UAs within the Future Combat System (FCS)--the network servicing echelons at levels below those served by WIN-T. The figure also shows that WIN-T, as an extension of the Global Information Grid (GIG), provides "reachback" to the continental United States (CONUS). Other external interfaces to current, Joint, Allied, and Coalition forces--the networks adjacent to WIN-T--are also depicted in Figure 2.
WIN-T will transition the Army from a semi-mobile, switched, digital channel system to a fully mobile network that is based on the Internet Protocol (IP) and provides integrated voice, data, and video. It will provide automated, ad hoc management "on the move" with management and services in support of, and controlled by, the commander's policy.
2.4 Contractual Aspects of the Software Architecture Evaluation
WIN-T has been designated an Acquisition Category (ACAT) 1D program, which is the acquisition category for programs in excess of $2.19 billion in procurement funding. WIN-T will actually be well in excess of that amount and, as of this writing, is in the System Design and Development Phase between Milestones B and C. These milestones are decision points on the acquisition path with Milestone B being the decision to proceed with systems design and development and Milestone C being the decision to begin production. The WIN-T development effort was awarded initially as two separate, competing contracts to General Dynamics and Lockheed Martin with the intent to down select to one contractor at about the time of Milestone C. Two years after contract award (and approximately one year after Milestone B), WIN-T underwent a contractual transition that resulted in the elimination of the "competitive fly-off" arrangement. For a variety of reasons, the individual contractual efforts were merged under a single contract with General Dynamics as the prime contractor and Lockheed Martin as a major subcontractor responsible for approximately 50% of the effort. Within this structure, known as the Combined Team, Lockheed Martin has the lead for software development and integration. Each contractor has a software architect, and the General Dynamics architect is designated as the lead software architect. This new single-contract arrangement resulted in a larger number of development contractors participating in the architecture evaluation. It also enabled the stakeholder interests of both major contactors to be accommodated and promoted closer collaboration within the Combined Team.
It was in this context that the Army's Strategic Software Improvement Program (ASSIP) decided to provide funding for a software architecture evaluation of WIN-T, which was to be led by the SEI and based on the ATAM (described in the next section).
Because a software architecture evaluation was not part of the existing WIN-T contract, the PD had to approve the evaluation before proceeding. The government's chief software architect met the PD, WIN-T and explained the process and benefits of an ATAM evaluation. The PD subsequently approved the ATAM evaluation, provided that the contractors were willing to support this effort and that the contractual costs associated with conducting the software architecture evaluation were not excessive.
The next step was to obtain the cooperation of the contractors. Information was provided to the managers and leaders of both contractor organizations via phone, email, and links to the ATAM section of the SEI Web site regarding the process, benefits, and output of an ATAM evaluation. Both contractor organizations were equally enthusiastic about performing an ATAM evaluation.
Before final approval could be obtained, the cost and schedule impact of diverting effort from the planned schedule of events to allow for the ATAM evaluation to take place had to be addressed. With regard to the impact, all the affected parties felt that the potential return of conducting a software architecture evaluation using the ATAM more than justified altering the planned schedule of events and the additional work required on the part of the participating contractors and WIN-T stakeholders. The other costs were for labor, travel, and lodging to have the contractor stakeholders prepare for and participate in the ATAM evaluation. The cost for travel and lodging was estimated at $2,000. The contractual effort to prepare and participate in the ATAM evaluation was estimated to be 200 hours. This included time to prepare and present the required briefings and the participation time spent by the lead architect, four software engineers, and two system engineers representing both contractor organizations.
The KO, WIN-T modified the existing Task Execution Plan (TEP) to support the evaluation. The changes to the TEP did not include the cost of developing and documenting the software architecture, as those tasks were already contract requirements. In fact, the first draft of the Combined Team software architecture had just been delivered. The PD felt that the schedule and cost impact were entirely reasonable and affordable for an ACAT-1D program and gave final approval to proceed with the evaluation.
Once approval to proceed was granted, arrangements were made to have stakeholders from internal offices within the PM (e.g., eventual software maintainers from the CECOM SEC) and stakeholders from external agencies (e.g., TRADOC TSM) participate in the ATAM to ensure that the interests of the WIN-T end users were adequately represented. No difficulties were encountered, and all stakeholders were enthusiastic about the opportunity to have their concerns and architectural issues aired.
3 The ATAM]
[6 Conclusion]
3 The ATAM
The purpose of the ATAM is to assess the consequences of architectural decision alternatives in light of quality attribute requirements [Kazman 00]. The major goals of the ATAM are to
- elicit and refine a precise statement of the architecture's driving quality attribute requirements
- elicit and refine a precise statement of the architectural design decisions
- evaluate the architectural design decisions to determine if they address the quality attribute requirements satisfactorily
The ATAM is predicated on the fact that an architecture is suitable (or not suitable) only in the context of specific quality attributes that it must impart to the system. The ATAM uses stakeholder perspectives to produce a collection of scenarios that define the qualities of interest for the particular system under consideration. Scenarios give specific instances of usage, performance requirements, growth requirements, various types of failures, various possible threats, and various likely modifications. Once the important quality attributes are identified in detail, the architectural decisions relevant to each one can be illuminated and analyzed with respect to their appropriateness.
The steps of the ATAM are carried out in two main phases. In the first phase, the evaluation team interacts with the system's primary decision makers: the architect(s), manager(s), and perhaps a marketing or customer representative. During the second phase, a larger group of stakeholders is assembled, including developers, testers, maintainers, administrators, and users. The two-phase approach insures that the analysis is based on a broad and appropriate range of perspectives.3
Phase 1:
- Present the ATAM. The evaluators explain the method so that those who will be involved in the evaluation have an understanding of the ATAM process.
- Present business drivers. The appropriate system representatives present an overview of the system, its requirements, business goals, context, and the architectural quality drivers.
- Present architecture. The system or software architect (or another lead technical person) presents the architecture.
- Catalog architectural approaches. The system or software architect presents general architectural approaches to achieve specific qualities. The evaluation team captures a list and adds to it any approaches they saw during Step 3 or learned during their pre-exercise review of the architecture documentation. For example, "a cyclic executive is used to ensure real-time performance." Known architectural approaches have known quality attribute properties that will help in carrying out the analysis steps.
- Generate a quality attribute utility tree. Participants build a utility tree, which is a prioritized set of detailed statements about what quality attributes are most important for the architecture to achieve (such as performance, modifiability, reliability, or security) and specific scenarios that express these attributes.
- Analyze architectural approaches. The evaluators and the architect(s) map the utility tree scenarios to the architecture to see how it responds to each one.
Phase 2:
Phase 2 begins with an encore of the Step 1 ATAM presentation and a recap of the results of Steps 2 through 6 for the larger group of stakeholders. Then these steps are followed:
- Brainstorm and prioritize scenarios. The stakeholders brainstorm additional scenarios that express specific quality concerns. After brainstorming, the group chooses the most important ones using a facilitated voting process.
- Analyze architectural approaches. As in Step 6, the evaluators and the architect(s) map the high-priority brainstormed scenarios to the architecture.
- Present results. A presentation is produced that captures the results of the process and summarizes the key findings that are indicative of what will be in the final report (a product of Phase 3).
Scenario analysis produces the following results:
- a collection of sensitivity and tradeoff points. A sensitivity point is an architectural decision that affects the achievement of a particular quality. A tradeoff point is an architectural decision that affects more than one quality attribute (possibly in opposite ways).
- a collection of risks and non-risks. A risk is an architectural decision that is problematic in light of the quality attributes that it affects. A non-risk is an architectural decision that is appropriate in the context of the quality attributes that it affects.
- a list of current issues or decisions not yet made. Often during an evaluation, issues not directly related to the architecture arise. They may have to do with an organization's processes, personnel, or other special circumstances. The ATAM process records these issues, so they can be addressed by other means. The list of decisions not yet made arises from the stage of the system life cycle during which the evaluation takes place. An architecture represents a collection of decisions. Not all relevant decisions may have been made at the time of the evaluation, even when designing the architecture. Some of these decisions are known to the development team as having not been made and are on a list for further consideration. Others are news to the development team and stakeholders.
Results of the overall exercise also include the summary of the business drivers, the architecture, the utility tree, and the analysis of each chosen scenario. All of these results are recorded visibly so all stakeholders can verify that they have been identified correctly.
The number of scenarios analyzed during the evaluation is controlled by the amount of time allowed for the evaluation, but the process insures that the most important ones are addressed.
After the evaluation, the evaluators write a report documenting the evaluation and recording the information discovered. This report also documents the framework for ongoing analysis discovered by the evaluators. Clements, Kazman, and Klein provide detailed descriptions of the ATAM process [Kazman 00, Clements 02b].
[3 The ATAM]
[6 Conclusion]
4 The ATAM Evaluation of WIN-T
4.1 Background
The liaison between the ATAM evaluation team leader and the WIN-T project was the government's lead software engineer for WIN-T. Together, they coordinated the dates for the Phase 1 and Phase 2 meetings, agreed on which stakeholders to invite to each, worked out the delivery of documentation to the team for pre-evaluation review, and worked to make sure that the Step 2 (business drivers) and Step 3 (architecture) presentations were prepared and contained the appropriate information.
Phase 1 took place on February 1-2, 2005 and was followed by Phase 2 on February 8-9, 2005. The evaluation team consisted of four members from the SEI, plus two Army members who had qualified for ATAM participation by successfully completing ATAM training and receiving the SEI ATAM Evaluator Certificate. The evaluation team members' organizations and roles are shown in Table 1.4
Table 1: Evaluation Team Members
Army, Research, Development, and Engineering Command (RDECOM) CERDEC Software Engineering Directorate (SED) |
The system stakeholders (architects, managers, developers, testers, integrators, etc.) participating in the WIN-T ATAM evaluation exercise are identified in Table 2 and Table 3.
Table 2: Attendees for Phase 1 of the WIN-T ATAM Evaluation
Table 3: Attendees for Phase 2 of the WIN-T ATAM Evaluation
PM, WIN-T (Space and Terrestrial Communications Directorate [S&TCD]) |
|
4.2 Business and Mission Drivers
Step 2 of the ATAM method is a presentation of the system's business and mission drivers. Before the exercise, the evaluation leader works with the person making the presentation and provides him or her with a standard presentation outline and template, to make sure the desired information is produced. The goal of the one-hour presentation is to understand why (from the development side as well as the acquisition side) this system is being created. For government acquisition programs, the person making the presentation is usually from the program office for the system being acquired. The purpose is to start collecting quality attribute goals against which the architecture can be evaluated.
For the WIN-T evaluation, the Army's PD, WIN-T gave an overview of WIN-T and described the Army's business objectives for the program. The driving business and mission requirements for the WIN-T products and the architecture goals derived from these requirements included the points described below.
4.2.1 General Points
- The purpose of the WIN-T system is to provide a single integrating communications network to (1) function as the Army's tactical portion of the GIG and (2) to link FCS with higher Army echelons and the GIG.
- WIN-T is to provide a reliable and secure network with high speed, high capacity, and high quality of service (QoS).
- The WIN-T system is required to support mobile communications and provide a replacement communications architecture for legacy systems such as MSE, TRI-TAC, Integrated Systems Control (ISYSCON), and Trojan Spirit.
- The Win-T deployment level is Theater to Maneuver Battalion. It will be owned, operated, and maintained by both signal and non-signal units.
- The development schedule is aggressive with the Initial Operational Test (IOT) scheduled for September 8, 2005. Spiral development will be used to achieve the initial capability.
4.2.2 Points Related to Business Goals
- Deploy a user-friendly human machine interface (HMI) for the WIN-T operator.
- Minimize the amount of network management the WIN-T operator must perform (automate the planning process to the maximum extent possible).
- WIN-T must be operable and maintainable within the budget and schedule.
- WIN-T must be transparent to the combat users.
- The operation of the software must be transparent to the WIN-T operators.
- The maintenance of the software must be as transparent as possible to the WIN-T operators.
4.2.3 Points Related to Key Performance Parameters
Key performance parameters for WIN-T are listed in Table 4.
Table 4: Key Performance Parameters (KPPs) for WIN-T
4.2.4 Points Related to Business Constraints
Business constraints affecting WIN-T include
- long life cycle
- interoperable
- Army Enterprise network (echelon above WIN-T)
- FCS (echelon below WIN-T)
- Joint networks
- Coalition network
- upgradeable
4.3 Architecture Presentation
Before the evaluation, the evaluation leader works with the architect to prepare an architecture presentation for the evaluation exercise. The presentation lasts between one and two hours and focuses on (1) a set of architectural views and (2) the primary approaches used to satisfy the architecturally significant requirements and driving quality attributes.
The views presented included
- a context view
- a layered view
- a deployment view
- a functional or module decomposition view
- one or more component-and-connector views
4.3.1 Context View
The context view (Figure 2) puts WIN-T into perspective with respect to its users and other participating elements.
Figure 2: Context Diagram (OV-1)
4.3.2 Layered View
The layered view is shown in Figure 3. The layer concept is used to help bring properties of modifiability and portability to a software system. A layer is an application of the principle of information hiding.
Figure 3: Layered View
4.3.3 Deployment View
Figure 4 shows a deployment view that illustrates the allocation of software to hardware.
Figure 4: Deployment View
4.3.4 Functional or Module Decomposition View
The functional view for WIN-T has six computer software configuration items (CSCIs) with the relationships among them shown in Figure 5. The NetOps (Network Operations) functional area includes NetOps Management (NM), Information Assurance (IA) management, Information Dissemination Management (IDM), and Network Services (NS), all running in the Operational Environment (OE). The Transmission Subsystem (TS) is software and firmware that runs on the WIN-T radios.
Figure 5: Functional View
Module decomposition views can consist of several levels of decomposition; the finest grained levels indicate the smallest units of implementation that might be affected by a modification to the system. Figure 6 is an example hierarchical functional decomposition for one CSCI. In this figure, Training is shown as a descendant of the CSCI, whereas it is actually a descendant of the unit labeled "Operational Environment."
Figure 6: Decomposition View
4.3.5 Component-and-Connector View
One or more component-and-connector views show the runtime interaction of computational elements and communication pathways. Figure 7 shows how the major functional elements communicate via a message bus.
Figure 7: Component-and-Connector View
Figure 8 shows the replication/failover scheme for fault tolerance and high availability.
Figure 8: Replication/Failover Scheme
4.4 Architectural Approaches
After the architecture is presented, the evaluation team summarizes the architectural approaches that have been employed. The team compiles the list by reviewing the architecture documentation ahead of time and extracting the approaches mentioned in the architecture presentation.
Although the WIN-T architecture employs many approaches, the five main approaches described in Table 5 were identified.
Table 5: Architectural Approaches
These architectural approaches are illustrated in Figure 9.
Figure 9: Architectural Approaches
4.5 Utility Tree
The utility tree provides a vehicle for translating the quality attribute goals articulated in the business drivers presentation to scenarios that express quality attribute requirements in a form specific enough for them to be "tested" against the architecture. The stakeholders who are present at Phase 1 construct the utility tree under the facilitated guidance of the ATAM team leader.
In this tree, "Utility" is the root node and expresses the overall "goodness" of the system. In the case of WIN-T, the second level nodes were modularity, adaptability, resilience, usability, security, performance, interoperability, information dissemination, autonomous operation, standards compliance, scalability, flexibility, maintainability, supportability, survivability, mobility, and affordability.
Under each of these quality attributes are specific concerns. These concerns arise from considering the quality-attribute-specific stimuli and responses that the architecture must address. For example, in the case of WIN-T, modularity was decomposed into the following concerns:
- replacement of an architectural component
- loose coupling/high cohesion
- separately implementable
- well-defined interfaces
- designed from a common set of components
Finally, these concerns are characterized by a small number of scenarios. These scenarios become leaves of the utility tree; thus the tree has four levels.
A scenario represents a use, or modification, of the architecture, applied not only to determine if the architecture meets a functional requirement but also (and more significantly) to predict system qualities such as performance, reliability, modifiability, and so forth.
The scenarios at the leaves of the utility tree are prioritized along two dimensions:
These nodes are prioritized relative to each other using ranking pairs of High, Medium, and Low (H, M, L), where the first value in the pair indicates the degree of importance to the system and the second indicates the degree of difficulty for the architecture to achieve it.
The quality attribute utility tree elicited during the WIN-T evaluation is reproduced in Table 6. It is typical in size to utility trees collected at other ATAM-based evaluations. Not all the attribute concerns are filled in with elaborating scenarios. This is normal and reflects the fact that sometimes stakeholders can think of a broad description of a quality attribute but not a specific requirement for it. The scenarios listed in Table 6 are numbered in the order in which they were created.
Table 6: WIN-T Quality Attribute Utility Test5
4.6 Scenario Generation and Prioritization
In addition to the scenarios at the leaves of the utility tree, the scenario elicitation process in Step 7 allows the larger group of stakeholders to contribute additional scenarios that reflect their concerns and understanding of how the architecture will accommodate their needs. While the scenarios that appear in the utility tree are developed top down, the scenarios generated by the larger group of stakeholders are developed bottom up. The combination of approaches provides some assurance that high-priority scenarios are surfaced. A particular scenario may, in fact, have implications for many stakeholders: for a modification, one stakeholder may be concerned with the difficulty of a change and its performance impact, while another may be interested in how the change will affect the "integrability" of the architecture.
Table 7 shows the scenarios that were collected by a round-robin brainstorming activity during Step 7 in Phase 2 of the ATAM evaluation. Each scenario is elicited in three parts: (1) a stimulus, describing an interaction of a stakeholder with the system, (2) an environment, describing what activity was ongoing at the time of the stimulation, and (3) a response, indicating the desired outcome, in quantitative terms. Stakeholders were free to choose a scenario from the utility tree as their contribution or create one on their own. After the scenarios were generated, they were prioritized using a voting process in which participants were given 10 votes that they could allocate to any scenario or scenarios they chose. The number of votes each scenario received is shown in the rightmost column of the table. Numbering begins at 49, since there were 48 scenarios previously collected in the utility tree.
Table 7: Phase 2 Scenarios
A COTS product is becoming increasingly incompatible with other COTS products being used. |
The COTS product is replaced.1 |
|||
WIN-T has the requirement to interface with a lower level system (ISYSCON, TIM). |
||||
A non-signal operator/ maintainer needs to identify and replace a faulty piece of equipment. |
WIN-T provides an automated capability to assist in troubleshooting at non-signal user level. |
|||
A WIN-T gateway node has been corrupted due to a network intrusion. |
Intrusion and corruption is detected and another node takes over duties. |
|||
A legacy application replaces an organic service with a WIN-T service. |
All documentation and APIs are sufficient; service implementations are available for inclusion in an integration/test environment. |
|||
Problem is detected, isolated, logged; information is conveyed to maintainers for development of a patch. |
||||
High-priority information needs to be moved across the network, but the current setup prohibits the timely transfer. |
WIN-T has to reconfigure itself within the current policies to allow the transfer to occur; reconfigures back at completion |
|||
Signal officer gets an OPORD that requires building a network. |
WIN-T provides planning tools to generate the network and the signal annex within required time. |
1 During voting, this scenario was combined with scenario #73. |