Scenario-Based Analysis of Software Architecture
- [Top] [Prev] [Next]
[Bottom]
Scenario-Based Analysis of Software Architecture
Rick Kazman
Department of Computer Science, University of Waterloo
Waterloo, Ontario
kazman@sei.cmu.edu
Gregory Abowd
College of Computing, Georgia Institute of Technology
Atlanta, Georgia
Gregory.Abowd@cc.gatech.edu
Len Bass
Software Engineering Institute, Carnegie Mellon University
Pittsburgh, Pennsylvania
ljb@sei.cmu.edu
Paul Clements
Software Engineering Institute, Carnegie Mellon University
Pittsburgh, Pennsylvania
pclement@sei.cmu.edu
Abstract: Software architecture is one of the most important tools for designing and understanding a software system, whether that system is in preliminary design, active deployment, or maintenance. Scenarios are important tools for exercising an architecture to gain information about a system's fitness with respect to a set of desired quality attributes.This paper presents a set of experiential case studies that illustrate the methodological use of scenarios to gain architecture-level understanding and predictive insight into large real-world systems in various domains. A structured method for scenario-based architectural analysis, called SAAM, is discussed. SAAM uses scenarios to analyze architectures with respect to achieving quality attributes. Finally, lessons and morals are presented, drawn from the growing body of experience in applying scenario-based architectural analysis techniques.
Keywords: Software Architecture; Software Analysis Methods; Software Quality, Software Architecture Analysis, Applications of Scenarios
-
1 Introduction
Analysis of a proposed software system to determine the extent to which it meets desired quality criteria is beneficial yet difficult. Some of the reasons why such analysis is difficult include an inadequate understanding of both the system's high level design and the quality attributes that it is to achieve. With the recent surge of interest in software architecture,1 some of the issues involved in the high-level design of software systems are being clarified. Our goal in this paper is a demonstration of how to exploit software architectural concepts to analyze2 complex software systems with respect to their quality attributes. We compensate for the lack of fundamental understanding about how to express these attributes by using scenarios to capture essential actions involving the system under analysis.
Scenarios are brief narratives of expected or anticipated use of a system from both development and end-user viewpoints. A structured method employing scenarios to analyze architectures is the Software Architecture Analysis Method (SAAM). SAAM is described in Section 2. Experience with SAAM and SAAM-related techniques is recounted in Section 3. Section 4 explores lessons learned.
We begin with a discussion of the relationship among software architecture, quality attributes, and scenarios.
1.1 Software architecture
Software architecture describes a high-level configuration of components that compose the system, and the connections that coordinate the activities of those components. We say software architecture here, but it is quite often the case that such high-level configurations describe the functions that are ultimately performed by either software or hardware components. We also say a high-level configuration rather than the high-level configuration, because a system can be composed of more than one type of component; each decomposition, therefore, has its own configuration. For instance, a system may be composed of a set of modules, and a set of cooperating sequential processes, each residing in one or more modules. Both viewpoints are valid, and both are architectural in nature. But they carry different information.
From the process viewpoint, we can describe the interaction of system processes during execution, in terms of how and when processes become active or dormant, pass or share data, or synchronize. From the module viewpoint, we can describe the interaction of the teams responsible for building the modules, in terms of the information they are: allowed to share, required to share (interfaces), or prohibited from sharing (implementation secrets). The process viewpoint has implications for performance. The module viewpoint has implications for maintainability. This sharp distinction between run-time versus development-time descriptions and properties is a recurring theme in our work.
Issues in software architecture are, by and large, not new. They date back at least to 1968 when Dijkstra pointed out it that pays to consider how to structure a computer program, in addition to considering how to make it compute the correct answer [4]. People who build large computer-based systems have been analyzing the allocation of function onto configurations for a long time.
Many software engineering textbooks describe the development stage between requirements and detailed design as "architectural design". This stage is compatible with our notion of when the definition of the software architecture occurs, in the transition from problem definition to solution space. Even though the ideas and motivations underlying software architecture are not novel, it is only within the past few years that researchers and practitioners have made explicit the architectural issues in their work, and begun to worry about the representation of the architecture as an important and living artifact in its own right within the life cycle of a product. Even more recent is the notion that an architectural representation may be profitably analyzed to determine how appropriate it is for its intended use.
Software architecture manifests its usefulness in the software development life cycle in the following ways:
- An architecture is often the first artifact in a design that
represents decisions on how requirements of all types are to be achieved. As
the manifestation of early design decisions, the architecture represents those
design decisions that are hardest to change and hence require the most careful
consideration.
- An architecture is the key artifact in achieving successful
product line engineering, the disciplined structured development of a family of
similar systems with less effort, expense, and risk than developing each system
independently [13].
- Architecture is usually the first artifact to be examined
when a programmer (particularly a maintenance programmer) unfamiliar with the
system begins to work on it.
Our emphasis on analysis of software architectures is compatible with the belief that understanding of the implications of a design leads to early detection of errors and to more predictable and cost-effective modifications to the system over its entire life cycle.
1.2 Quality Attributes
We are interested in evaluating architectures to determine their fitness with respect to certain properties or qualities of the resulting system. These qualities fall into the following three categories3:
- qualities described by observing the output of the executing
system in the presence of some input. Besides correctness, these qualities
include those usually called by names such as security, reliability, and
availability. Some, such as performance and throughput, are time-dependent;
others are not.
- qualities described by measuring the activities of a
development or maintenance team. These include maintainability, portability,
adaptability, and scalability.
- qualities described by measuring the activities of a particular user (possibly another system) in concert with the executing system. These include usability, predictability, and learnability.
Though this categorization is a useful one, it is too difficult to analyze an architecture based on these abstract qualities. The qualities themselves are too vague and they provide very little procedural support for evaluating an architecture.
As an example of vagueness, suppose we can change the colors in a user interface by changing a resource file which is read at run-time, but changing the fonts in the interface requires a re-compilation. Is this system modifiable or not? The answer is perhaps "yes" with respect to changing colors, but "no" with respect to changing fonts. And, whether the design is acceptable or not depends on predictions of actual usage: if the user interface is modifiable in a way that is important to its owner, then we can say that the system is appropriately modifiable. This notion of appropriateness applies to all quality factors.
The lesson is that at the present time and for the foreseeable future, there are no simple (scalar) "universal" measurements for attributes such as safety or portability. Rather, there are only context-dependent measures, meaningful only in the presence of specific circumstances of execution or development. Safety benchmarks are a fine example. If there were a universal measurement of safety, benchmarks would be unnecessary. As it is, a benchmark represents data about a system executing with particular inputs in a particular environment.
While we may wish for better understanding and more universal expression of quality attributes, for now we must recognize the role played by specifying a particular operational context for a system. To represent contexts, we use scenarios.
-
1.3 Scenarios
Scenarios have been widely used and documented in object-oriented design and user interface design as a technique to elicit and validate requirements, particularly for the operator of the system ([6], [9]). Scenarios have also been widely used as a method of comparing design alternatives. Programmers often use them to understand an already-built system, by asking how the system responds (component by component) to a particular input or operational situation.
Scenarios have not, however, been used as a tool to analyze quality, our primary utilization of them. We use scenarios to express the particular instances of each quality attribute important to the customer of a system. We then analyze the architecture under consideration with respect to how well or easily it satisfies the constraints imposed by each scenario.
Scenarios differ widely in breadth and scope. We use scenarios as a brief description of an anticipated or desired use of a system. At this point in our work, scenarios are typically one sentence long and could more appropriately be called vignettes.
We emphasize the use of scenarios appropriate to all roles involving a system. The operator role is one widely considered, but we also have roles for the system designer and modifier, the system administrator, and others, depending on the domain. When analyzing a system all roles relevant to that system must be considered since design decisions are made to accommodate all roles. This analysis of roles leads once again to the distinction between run-time (e.g., a scenario for the operator role) and development-time (e.g., a scenario for a maintenance engineer).
The process of choosing scenarios for analysis forces designers to consider future uses of, and changes to, the system. Thus, we believe that architectural analysis cannot give precise measures or metrics of fitness. Such measures need to be couched in terms of qualities (e.g. "how modifiable is this architecture?") and such questions are typically of little value. What we really want to know is: "how does this architecture accommodate the following change?" or "how does this architecture accommodate a change of the following class?", and we use architectural analysis to guide our inspection of the architecture. The analysis process focuses attention on potential trouble areas.
There is a distinction between scenario types that is important in our work. Recall that a scenario is a brief description of an anticipated or desired use of a system. It may be the case that the system directly supports that scenario, meaning this use requires no modification to the system in order to be performed. This would usually be determined by demonstrating how the existing architecture would behave in performing the scenario (rather like a simulation of the system at the architectural level).
If a scenario is not directly supported, that means that there must be some change to the system that we can represent architecturally. This change could be a change to how one or more components perform an assigned activity, the addition of a component to perform some activity, the addition of a connection between existing components, or a combination of these. We refer to the first class of scenarios as direct scenarios and the second class as indirect scenarios. We will use direct and indirect scenarios at different stages in our analysis method.
To illustrate the distinction between direct and indirect scenarios, consider the following simple example, where we are attempting to evaluate two different calculator applications, A and B, each of which has a graphical user interface. We have two scenarios which we will use to evaluate calculators A and B.
Scenario 1: resize the calculator and have all of its components (buttons, display area, fonts) scale appropriately.
Scenario 2: switch between standard infix notation and RPN (Reverse Polish Notation).
Calculator A already supports scenario 1. That is, in its existing state it supports resize with scaling. Calculator A does not support RPN however; satisfying scenario 2 will require modifying calculator A's code in some way. Thus, scenario 2 is indirect with respect to calculator A.
The situation is precisely the reverse with calculator B. It was created to have a fixed size and geometry but supports both notations and allows the user to freely toggle between notation types at run-time. Scenario 2 can be satisfied by executing calculator B, and is thus direct,while scenario 1 requires modifying B's implementation, and is thus indirect. These cases are summarized in Table 1.
Table 1: Scenario categorizations with respect to different products
Product
Scenario 1
Scenario 2
One final point; not all scenarios are impacted by architecture-level decisions. For example, a portability scenario might have architectural implications (such as determining how functionality should be divided) but it may also depend upon code-level or hardware-level factors, such as byte ordering. Furthermore, some scenarios simply cannot be evaluated using architectural information. For example, if a developer's scenario was to ensure that no module had more than 250 lines of code, this constraint could neither be checked or ensured by architecture-level analysis.
1.4 How Analysis Influences Design
Analysis and design are closely intertwined. That is, when a designer knows the basis for an upcoming design review, the design itself is affected positively. Thus, it is no surprise that when applying an analysis method, there are a number of influences on the designers independent of the analysis itself. In carrying out scenario-based architectural analysis, we have observed the following benefits:
- the ability to compare competing high-level designs for a system and
document those comparisons
- focussing design activity where it is needed most, and can be reused most
- enhancing high-level communications among development team members and
between the developers and customers of a system
We see specific examples of these benefits in the case studies presented. At the current level of maturity of the discipline of software architecture, the benefits which architectural analysis brings to a development project are mostly people- and process-oriented. That is, the output of an analysis activity calls attention to a particular problem in the design or the communication of the design; it does not propose solutions to the problems identified. This situation will change as our models and analysis techniques improve, although consideration of people- and process-oriented issues will remain crucial.
Architectural analysis helps improve communication among development team members, and between team members and "outsiders" (upper-level managers, clients, users). Part of this improved communication is accrued simply by choosing a common syntactic and semantic notation for architectural representation. A much larger part of the improved communication, however, arises because scenario-based software architecture analysis helps to focus high-level and global software design discussions on specific problem areas. This analysis motivates development teams to critically evaluate and discuss the architectural alternatives of their system early in the life cycle. How this focus is achieved-through effective use of scenarios-is the subject of the rest of this paper.
1 See, for example, the April, 1995, special issue of IEEE Transactions on Software Engineering devoted to software architecture. 2 We distinguish between analysis and evaluation. Analysis is, according to Webster's New Collegiate dictionary, "1. separation of a whole into its component parts 2. an examination of a complex, its elements, and their relations." This definition is at the heart of what we consider important in examining software architectures. Evaluation pre-supposes a particular value scale or system, which in our world is not always forthcoming. 3 Although our categorization of requirements is novel, their identification is not. Most software engineering texts now counsel designers to plan and account for life-cycle properties in addition to correctness, and teach the importance of quantitative (testable) specifications of such properties. Our work extends that trend.
[Top] [Prev] [Next] [Bottom]
Feedback: kazman@sei.cmu.edu
- the ability to compare competing high-level designs for a system and
document those comparisons
- An architecture is often the first artifact in a design that
represents decisions on how requirements of all types are to be achieved. As
the manifestation of early design decisions, the architecture represents those
design decisions that are hardest to change and hence require the most careful
consideration.
-