How do you know if a software architecture for a system is suitable without having to build the system first? The answer is to conduct an evaluation of it. A formal software architecture evaluation should be a standard part of the architecture-based software development life cycle. Architecture evaluation is a cost-effective way of mitigating the substantial risks associated with this highly important artifact.

The creation and maintenance of a complex software-intensive system involves making a series of business-critical architecture design decisions. The SEI Architecture Tradeoff Analysis Method (ATAM) provides software architects with a framework for understanding the technical tradeoffs they face as they make design or maintenance decisions. But the biggest tradeoffs in large complex systems usually have to do with economics, and the ATAM does not provide any guidance for understanding these economic tradeoffs. Organizations need to know how to invest their resources to maximize their gains, meet their schedules, and minimize their risks. When economics have been addressed in the past, the focus has usually been on costs, and even then only the costs of building the system have been considered, not the long-term costs of maintenance and upgrade. Yet the benefits that an architectural decision may bring to an organization are as important—or perhaps even more important—than the costs. Clearly we need to consider both, that is to consider the return on investment (ROI) of any architectural decision. Because the resources for building and maintaining a system are finite, there must be a rational process for choosing among architectural options, during the initial design phase and during subsequent periods of upgrade. These options will have different costs, consume different amounts of resources, implement different features (each bringing some benefit to the organization), and have some inherent risk or uncertainty. To explore the effects of these options, economic software models are needed that take into account costs, benefits, risks, and schedule implications.

The ATAM uncovers the architectural decisions that are made (or are being considered) for the system, and links these decisions to business goals and quality attributes. The CBAM—Cost Benefit Analysis Method—builds on this foundation, as exemplified by the cubes labeled P, A, S, and M, in this figure (representing Performance, Availability, Security, and Modifiability respectively). These quality attribute decisions (and there may be many others, for other qualities) result in some benefit to the system's stakeholders. The CBAM guides system engineers and other stakeholders to determine the costs and benefits associated with the architectural decisions that result in the system's qualities. Given this information, the stakeholders can then reflect upon and choose among the potential architectural decisions. For example, they could decide whether to use redundant hardware, checkpointing, or some other method to address concerns about the system’s reliability. Or the stakeholders could choose to invest their finite resources in some other quality attribute—perhaps believing that higher performance will have a better ROI.

The SEI has worked with NASA’s Earth Observing System Data Information System (EOSDIS) Core System (ECS) for the past 2 years, applying the CBAM to aid in making investment decisions for the project. The Earth Observing System is a constellation of NASA satellites that gathers data about the Earth for the U. S. Global Change Research Program and other scientific communities world­wide. By using the CBAM the ECS managers were able to order a set of architectural strategies based upon their predicted ROI. But the true benefits of the CBAM extend far beyond the qualitative outcomes. There have been palpable social and cultural benefits as well. The CBAM process provided a great deal of structure to what was largely unstructured discussions where requirements and architectural strategies and personal opinions are freely mixed together, and where stimuli and response goals are not clearly articulated. The CBAM process forced the stakeholders to make their scenarios clear in advance, to assign utility levels to specific response goals, and to prioritize scenarios based on the resulting determination of utility. The CBAM forced the stakeholders to address risks and their resulting effects explicitly, rather than simply stating an “unease” with a particular technical direction.

Additional Information

For consulting information about CBAM, visit our consulting pages. Direct technical questions about CBAM to us using the link in the For more information box at the bottom of this page.

Software Architecture Training and Publications

Training

Publications

Further Reading

Evaluating Software Architectures: Methods and Case Studies
Paul Clements, Rick Kazman, & Mark Klein

Integrating Software-Architecture-Centric Methods into Extreme Programming (XP)
Robert L. Nord, James E. Tomayko, & Rob Wojcik

Integrating the Architecture Tradeoff Analysis Method (ATAM) with the Cost Benefit Analysis Method (CBAM)
Robert L. Nord, Mario R. Barbacci, Paul Clements, Rick Kazman, Mark Klein, Liam O'Brien, James E. Tomayko

Making Architecture Design Decisions: An Economic Approach
Rick Kazman, Jai Asundi, & Mark Klein

Quantifying the Value of Architecture Design Decisions: Lessons from the Field
M. Moore, R. Kazman, M. Klein, & J. Asundi