CERT-SEI

Cost Benefit Analysis Method

The Cost Benefit Analysis Method (CBAM) is an architecture-centric method for analyzing the costs, benefits, and schedule implications of architectural decisions. It also enables assessment of the uncertainty surrounding judgments of costs and benefits, thereby providing a basis for informed decision making about architectural design/upgrade. The CBAM builds on the Architecture Tradeoff Analysis Method (ATAM), although an ATAM is not an absolute prerequisite.

Challenges

  • How do you go about taking economic considerations into account when designing or modifying a system architecture?
  • How do you account for the costs involved?
  • How do you characterize and compare the benefits that will accrue to various architectural strategies?
  • How can costs and benefits be traded off against quality attributes or functionality?
  • How can you characterize the uncertainties involved in your estimates?

Description

How do you know if a software architecture for a system is suitable without having to build the system first? The answer is to conduct an evaluation of it. A formal software architecture evaluation should be a standard part of the architecture-based software development life cycle. Architecture evaluation is a cost-effective way of mitigating the substantial risks associated with this highly important artifact.

The creation and maintenance of a complex software-intensive system involves making a series of business-critical architecture design decisions. The SEI ATAM provides software architects with a framework for understanding the technical tradeoffs they face as they make design or maintenance decisions. But the biggest tradeoffs in large complex systems usually have to do with economics, and the ATAM does not provide any guidance for understanding these economic tradeoffs. Organizations need to know how to invest their resources to maximize their gains, meet their schedules, and minimize their risks. When economics have been addressed in the past, the focus has usually been on costs, and even then only the costs of building the system have been considered, not the long-term costs of maintenance and upgrade. Yet the benefits that an architectural decision may bring to an organization are as important—or perhaps even more important—than the costs.

Clearly we need to consider both, that is to consider the return on investment (ROI) of any architectural decision. Because the resources for building and maintaining a system are finite, there must be a rational process for choosing among architectural options, during the initial design phase and during subsequent periods of upgrade. These options will have different costs, consume different amounts of resources, implement different features (each bringing some benefit to the organization), and have some inherent risk or uncertainty. To explore the effects of these options, economic software models are needed that take into account costs, benefits, risks, and schedule implications.
 
cbam_1The ATAM uncovers the architectural decisions that are made (or are being considered) for the system, and links these decisions to business goals and quality attributes. The CBAM builds on this foundation, as exemplified by the cubes labeled P, A, S, and M, in this figure (representing performance, availability, security, and modifiability respectively). These quality attribute decisions (and there may be many others, for other qualities) result in some benefit to the system's stakeholders. The CBAM guides system engineers and other stakeholders to determine the costs and benefits associated with the architectural decisions that result in the system's qualities. Given this information, the stakeholders can then reflect on and choose among the potential architectural decisions. For example, they could decide whether to use redundant hardware, checkpointing, or some other method to address concerns about the system’s reliability. Or the stakeholders could choose to invest their finite resources in some other quality attribute, perhaps believing that higher performance will have a better ROI.

The SEI has worked with NASA’s Earth Observing System Data Information System (EOSDIS) Core System (ECS), applying the CBAM to aid in making investment decisions for the project. The Earth Observing System is a constellation of NASA satellites that gathers data about the Earth for the U. S. Global Change Research Program and other scientific communities world-wide. By using the CBAM, the ECS managers were able to order a set of architectural strategies based on their predicted ROI. But the true benefits of the CBAM extend far beyond the qualitative outcomes. There have been palpable social and cultural benefits as well. The CBAM process provided a structure to what were largely unstructured discussions where requirements and architectural strategies and personal opinions were freely mixed together, and where stimuli and response goals were not clearly articulated. The CBAM process forced the stakeholders to make their scenarios clear in advance, to assign utility levels to specific response goals, and to prioritize scenarios based on the resulting determination of utility. The CBAM forced the stakeholders to address risks and their resulting effects explicitly, rather than simply stating an “unease” with a particular technical direction.

The CBAM consists of the following steps:

  1. choosing scenarios and architectural strategies
  2. assessing quality attribute benefits
  3. quantifying the benefits of architectural strategies
  4. quantifying the costs and schedule implications of architectural strategies
  5. calculating desirability
  6. making decisions

Through the CBAM exercise, the CBAM team guides the stakeholders to determine a set of architectural strategies that address their highest priority scenarios. These chosen strategies furthermore represent the optimal set of architectural investments. They are optimal based on considerations of benefit, cost, and schedule, within the constraints of the elicited uncertainty of these judgments and the willingness of the stakeholders to withstand the risk implied by uncertainty.

Benefits

The CBAM enables users to make informed decisions about software requirements and software investments based on an analysis of the economic and architectural implications of those decisions.

Who Would Benefit

Software product or project managers, software product line managers, business analysts, software architects/senior designers.

Availability

SEI staff will work with a customer team to apply the CBAM as needed or answer technical questions about CBAM. Contact us using the link in the For more information box at the bottom of this page.

Software Architecture Training and Publications

Software Architecture Training at the SEI

Software Architecture Publications

Further Reading

Evaluating Software Architectures: Methods and Case Studies
Paul Clements, Rick Kazman, & Mark Klein

Integrating Software-Architecture-Centric Methods into Extreme Programming (XP)
Robert L. Nord, James E. Tomayko, & Rob Wojcik

Integrating the Architecture Tradeoff Analysis Method (ATAM) with the Cost Benefit Analysis Method (CBAM)
Robert L. Nord, Mario R. Barbacci, Paul Clements, Rick Kazman, Mark Klein, Liam O'Brien, James E. Tomayko

Making Architecture Design Decisions: An Economic Approach
Rick Kazman, Jai Asundi, & Mark Klein

Quantifying the Value of Architecture Design Decisions: Lessons from the Field
M. Moore, R. Kazman, M. Klein, & J. Asundi