|
||
|
||
| |
|
|
| Columns | The Architect | Volume
2 | Issue 4 | December 1999 |
|||||||||||||||||||||||
|
Read
previous Read
previous features
If
you would like
|
Software
Architecture Evaluation in the DoD Systems Acquisition Context Many modern defense systems rely heavily on software to achieve system functionality. Because software architecture is a major determinant of software quality, it follows that software architecture is critical to the quality of any software-intensive system. For a DoD acquisition organization, the ability to evaluate software architectures before these architectures are realized as finished systems can substantially reduce the risk that the delivered systems will not meet their quality goals. This column presents the basic principles of applying a software architecture evaluation in the DoD system acquisition context.
What Is Software Architecture?
It is important to understand that there is no such thing as the architecture of a system--that is, there is no single artifact that one can definitively point to as the architecture. There are, however, many relevant and important views of an architecture depending on the stakeholders and the system properties that are of interest. If we consider the analogy of the architecture of a building, various stakeholders such as the construction engineer, the plumber, and the electrician all have an interest in how the building is to be constructed. Although they are interested in different components and different relationships, each of their views is equally valid and is necessary to ensure that they will function properly together. Thus, all views are necessary to fully represent and fully analyze the architecture of the building. The analogy holds for a software architecture, but in this case the stakeholders might include the development organization, the end user, the system maintainer, the operator, and the acquisition organization. Each of these stakeholders has an important interest in different system properties. We will elaborate on the importance of software architecture to the delivered system and these various stakeholders next.
Why Is Software Architecture Important? The point is often made that the DoD buys systems not software,
so why should the DoD concern itself with software architectures?
Simply stated, almost all modern systems, including modern defense
systems, rely heavily on software to achieve critical functionality.
Thus, many important system quality goals--security, availability,
modifiability, performance, and so forth--are achieved through software.
The software architecture is a major determinant of software quality
and thus of system quality. So, even though the DoD is buying a
system, the software and in particular the software architecture
are of paramount importance in determining whether the DoD gets
the level of system qualities required. These interrelationships
are depicted in Figure 1 (adapted from Fisher
[98]). Figure
1: The Relationships Among System Quality It is also important to understand that architectures allow or preclude nearly all of the quality attributes of large, complex systems. For example, if your system has stringent performance requirements, then you must pay attention to things such as component interactions, communication mechanisms, scheduling policies, and component deadlines. If you have modifiability goals for your system, then you need to pay attention to encapsulation properties of your components. If reliability is important, then the architecture must provide schemes for redundancy, restart, fail-over. The list of such system-wide architectural concerns and strategies goes on and on. All of these approaches to achieving system quality are architectural in nature, having to do with the decomposition of the total system into parts and the ways in which those parts communicate and cooperate with each other. While a "good" architecture cannot guarantee a successful implementation (i.e., an implementation that meets its quality goals), a "bad" architecture can certainly preclude one, as shown in the many case studies in Software Architecture in Practice [Bass 98]. Additionally, architectural decisions are among the earliest design decisions made. If an inappropriate architectural choice is made, the consequences are profound. Studies show that the cost to fix an error found during requirements or early design phases are orders of magnitude less than the same error found during deployment or maintenance [Boehm 81]. Thus, it makes economic sense to take steps to ensure the quality of a software architecture. Next we will describe an approach that has proven successful in improving the quality of a software architecture.
Software Architecture Evaluation The SEI has been developing the Architectural Tradeoff Analysis MethodSM (ATAMSM) for the past two years [Kazman 99]. This method not only permits evaluation of specific architecture quality attributes but also allows engineering tradeoffs to be made among possibly conflicting quality goals. The ATAM draws its inspiration and techniques from three areas: the notion of architectural styles, the quality attribute analysis communities, and the Software Architecture Analysis Method (SAAM), which was the predecessor to the ATAM [Kazman 96]. The ATAM is intended to analyze an architecture with respect to its quality attributes, not its functional correctness. The ATAM involves a wide group of stakeholders including managers, developers, maintainers, testers, reusers, end users, and customers. It is meant to be a risk-mitigation method, a means of detecting areas of potential risk within the architecture of a complex, software-intensive system. This focus has several implications, including
What we aim to do in the ATAM, in addition to raising architectural awareness and improving the level of architectural documentation, is to record any risks, sensitivity points, and tradeoff points that we find when analyzing the architecture. Risks are architecturally important decisions that haven't been made (for example, the architecture team hasn't decided what scheduling discipline to use or whether to use a relational or object-oriented database), or decisions that have been made but whose consequences are not fully understood (for example, the architecture team has decided to include an operating system portability layer, but is not sure what functions should go into this layer). Sensitivity points are parameters in the architecture to which some measurable quality attribute is highly correlated. For example, it might be determined that overall throughput in the system is highly correlated to the throughput of one particular communication channel, and availability in the system is highly correlated to the reliability of that same communication channel. Finally, a tradeoff point is found in the architecture when a parameter of an architectural construct is host to more than one sensitivity point where the measurable quality attributes are affected differently by changing that parameter. For example, if increasing the speed of the communication channel mentioned above improves throughput but reduces its reliability, then the speed of that channel is a tradeoff point. To use ATAM effectively in the DoD, it is necessary to understand the special characteristics of the DoD acquisition environment.
The DoD Acquisition Management Process Context DoD 5000.2R prescribes a high-level acquisition process known as the DoD Acquisition Management Process. It serves as the overall roadmap for program execution and includes mandatory acquisition procedures and specific guidance for acquisition programs [Bergey 99]. Although the DoD management process is primarily directed toward major system-acquisition programs, it is intended to serve as a general model for all DoD acquisition programs. In particular, the contractual process that must be followed is prescribed in the Defense Federal Acquisition Regulation Supplement1 [DFARS 98]. This contractual process includes three important phases: the pre-award phase, the award phase, and the post-award phase. During the pre-award phase, the acquisition organization prepares and issues a request for proposal (RFP) and interested bidders may respond. During the award phase, source selection occurs. During the source-selection process, the acquisition organization evaluates proposals, obtains best and final offers from bidders, and selects a winning bidder. During the post-award phase, the government administers the contract, monitoring the technical progress and performance of the winning bidder. Depending on the scope of the contract, the winning bidder may or may not be responsible for support of the developed system following delivery and deployment. We will use these phases as a basis for describing different points at which the ATAM might be effectively applied in a DoD or government acquisition.
Applying Architecture Evaluation within the DoD Acquisition-Management Process Pre-Award and Award Phases for a System-Development Contract Two major activities that take place in the contractual pre-award and award phases are generation of an RFP and source selection respectively. Release of the RFP defines the official beginning of the solicitation period. After the solicitation formally closes, source selection commences with proposal evaluation and ends with a contract award. Specifying an ATAM-based architecture evaluation can be an effective means of evaluating the technical risks associated with a proposed software architecture. The results can be used as part of the technical evaluation criteria for source selection. The requirement to perform an ATAM-based architecture evaluation must be appropriately integrated into the RFP and the source-selection plan. Post-Award Contract Administration and Performance Phase for a System- Development Contract After contract award, an ATAM might be used for (at least) four purposes:
Other ATAM Contractual Applications The ATAM could also be applied in an acquisition for upgrading an existing system after it has been operationally deployed and is in its post-development/support life-cycle phase. Additionally, an ATAM could also be applied to a legacy system to evaluate and improve how well the architecture would support a set of proposed upgrades. Many legacy systems have only implicit software architectures: they were either never properly documented or the documentation has not been updated in concert with the system. In such cases, an architecture extraction and reconstruction activity would have to precede the ATAM to redocument the architecture [Kazman 98].
Conclusions While the ATAM is still in the developmental phase, it has already proven its ability to significantly improve software architectures in several pilot projects in a software development environment. The next challenge is to codify the application of ATAM principles in an acquisition environment. ATAM principles have been effectively applied to a limited extent in source selection, and the initial results are promising. The SEI is collaborating with several acquisition organizations on the use of the ATAM, to help them transition the process into their own organizations and to help them include appropriate language in an RFP to make architectural evaluation an integral part of evaluating proposals. As experience is gained in this area we will continue to share our lessons learned.
About the Authors Lawrence G. Jones is a senior member of the technical staff in the Product Line Systems Program of the Software Engineering Institute (SEI) of Carnegie Mellon University. In addition to his product line duties, he is also a member of the Capability Maturity Model® Integration (CMMI®) team. Before joining the SEI, he served in the United States Air Force in a variety of software development, management, and education positions. He is also the former chair of the Computer Science Department at the U.S. Air Force Academy. He holds a PhD in computer science from Vanderbilt University and master’s and bachelor’s degrees in industrial engineering from the University of Arkansas. Rick Kazman is a senior member of the technical staff at the SEI, where he is a technical lead in the Architecture Tradeoff Analysis Initiative. He is also an adjunct professor at the Universities of Waterloo and Toronto. His primary research interests within software engineering are software architecture, design tools, and software visualization. He is the author of more than 50 papers and co-author of several books, including a book recently published by Addison-Wesley entitled Software Architecture in Practice. Kazman received a BA and MMath from the University of Waterloo, an MA from York University, and a PhD from Carnegie Mellon University.
References
The views expressed in this article are the author's only and do not represent directly or imply any official position or view of the Software Engineering Institute or Carnegie Mellon University. This article is intended to stimulate further discussion about this topic. |
||||||||||||||||||||||||
 |
|
||||||||||||||||||||||||
| Copyright
© 1999 by Carnegie Mellon University. All rights reserved. |
|||||||||||||||||||||||||
|
|||||||||||||||||||||||||
