Software Engineering Institute Carnegie Mellon

Cost-Benefit Analysis Method (CBAM)

The creation and maintenance of a complex software-intensive system involves making a series of business-critical architecture design decisions. The Architecture Tradeoff Analysis Method (ATAM) provides software architects with a framework for understanding the technical tradeoffs they face as they make design or maintenance decisions. But the biggest tradeoffs in large complex systems usually have to do with economics, and the ATAM does not provide any guidance for understanding these economic tradeoffs. Organizations need to know how to invest their resources to maximize their gains, meet their schedules, and minimize their risks. When economics have been addressed in the past, the focus has usually been on costs, and even then only the costs of building the system have been considered, not the long-term costs of maintenance and upgrade. Yet the benefits that an architectural decision may bring to an organization are as important—or perhaps even more important—than the costs. Clearly we need to consider both, that is to consider the return on investment (ROI) of any architectural decision. Because the resources for building and maintaining a system are finite, there must be a rational process for choosing among architectural options, during the initial design phase and during subsequent periods of upgrade. These options will have different costs, consume different amounts of resources, implement different features (each bringing some benefit to the organization), and have some inherent risk or uncertainty. To explore the effects of these options, economic software models are needed that take into account costs, benefits, risks, and schedule implications.

The ATAM uncovers the architectural decisions that are made (or are being considered) for the system, and links these decisions to business goals and quality attributes. The CBAM – Cost Benefit Analysis Method – builds on this foundation, as exemplified by the cubes labelled P, A, S, and M, in the figure below (representing Performance, Availability, Security, and Modifiability respectively). These quality attribute decisions (and there may be many others, for other qualities) result in some benefit to the system's stakeholders. The CBAM guides system engineers and other stakeholders to determine the costs and benefits associated with the architectural decisions that result in the system's qualities. Given this information, the stakeholders can then reflect upon and choose among the potential architectural decisions. For example, they could decide whether to use redundant hardware, checkpointing, or some other method to address concerns about the system’s reliability. Or the stakeholders could choose to invest their finite resources in some other quality attribute—perhaps believing that higher performance will have a better ROI.

The SEI has worked with NASA’s Earth Observing System Data Information System (EOSDIS) Core System (ECS) for the past 2 years, applying the CBAM to aid in making investment decisions for the project. The Earth Observing System is a constellation of NASA satellites that gathers data about the Earth for the U. S. Global Change Research Program and other scientific communities world­wide. By using the CBAM the ECS managers were able to order a set of architectural strategies based upon their predicted ROI. But the true benefits of the CBAM extend far beyond the qualitative outcomes. There have been palpable social and cultural benefits as well. The CBAM process provided a great deal of structure to what was largely unstructured discussions where requirements and architectural strategies and personal opinions are freely mixed together, and where stimuli and response goals are not clearly articulated. The CBAM process forced the stakeholders to make their scenarios clear in advance, to assign utility levels to specific response goals, and to prioritize scenarios based on the resulting determination of utility. The CBAM forced the stakeholders to address risks and their resulting effects explicitly, rather than simply stating an “unease” with a particular technical direction.

Additional Information

Read More:

Products and Services: Learn about CBAM, and architecture evaluation and analysis related products and services.

Contact Information: For technical details about CBAM, contact Rick Kazman.

Working with the SEI: Learn more about working with the SEI in software architecture.