An assurance case can be developed incrementally throughout a development life cycle. An assurance plan defines the kinds of software assurance activities that will be accomplished at different phases of a project—in effect, it is a means to manage the assurance case development.
The assurance activities focus on the dependability attributes of a system, not its functional attributes, although deficiencies in functional attributes may affect system qualities such as reliability, availability, and safety. By and large, the dependability attributes of interest are attributes of the system as a whole (e,g., reliability or safety). These attributes depend on the interactions among system components. For assurance purposes, it is generally not sufficient to show that system components work correctly individually or in subsystems; it is also necessary to address component interactions to ensure a system's overall dependability attributes.
The assurance plan describes assurance case preparation activities undertaken to ensure that
- Requirements for dependability are understood, analyzed for conflicts that may require tradeoffs, and implemented by the development organization.
- Resources are allocated to production of the assurance case.
- Task assignments, skills, budget, and schedule are allocated appropriately.
- The assurance case remains accurate and current.
- The assurance case is presented effectively.
Our following examples show information that should appear in an assurance plan and an assurance case at different phases of a project's life cycle—namely, Requirements, Design, Integration and Evaluation, and Deployment. The use of these life-cycle terms does not imply a waterfall development model: our intent is to characterize how much is known about the system as its development evolves and what the implications would be for an assurance plan and case.
In the requirements phase, a description of the dependability attributes relevant to the project should be created. By the end of this phase, the plan will document (or reference) defined dependability requirements, and,in a general fashion, what activities will be undertaken in subsequent phases to demonstrate that the requirements will be or have been met. For example, the plan should focus on showing how
- The project's dependability requirements can be demonstrated within the available time and schedule.
- Requirements (including derived requirements) will be evaluated for consistency and completeness, especially for those requirements or areas likely to affect mission success or safety.
- The documents to be generated in this or later phases will support the assurance case for the project. For example, software functions whose failure would critically impact system dependability will need to receive particular attention in the fully developed assurance case. The plan should describe the approach that will be taken to identify these critical functions.
The assurance plan should also discuss staffing and scheduling issues related to achieving the dependability requirements and maintaining the plan/case.
As a design evolves, it will be possible to create a more meaningful assurance plan and case. For example, the assurance case (at the design stage) will typically address how the software has been structured to cope with various hazards introduced by the environment and possible software component failures. The plan should discuss the extent of the planned hazard analyses or software failure modes and effects analysis (FMEAs). The case will then reference these analyses to show how the hazards and potential failure modes have been mitigated by the design. If special analyses and simulations are to be undertaken to verify various parts of the system design, the nature of these should be described in the plan. The results of these analyses will then be referenced in the case as evidence supporting particular dependability claims.
Toward the end of the design phase, it should be possible to provide a fairly complete set of dependability claims that will be supported by evidence gathered during the implementation and integration/test phases. By developing such an assurance case structure at this point, it will be clearer what effort is needed later to demonstrate the validity of the claims and how this effort will be effective in providing such support.
For example, the strategy to be used for generating test cases is affected by how the test cases support the dependability claims. The strategy should be documented in the plan; the implementation of the testing strategy and the actual test case results will later be incorporated into the case.
Integration and Evaluation Phase
After integration and evaluation, the assurance case will show the detailed arguments that relate evidence gathered during this phase to the claims of assurance throughout the system. Typically the case will be augmented with test results.
For the assurance case to be useful, it must be maintained over time. The assurance plan for the deployment phase should discuss the processes and procedures to be used for keeping the assurance case current in the face of a changing system, possibly with multiple configurations. Examining the current case can help determine if proposed modifications will invalidate or change arguments and claims and, if so, which parts of the case will need to be updated. Further, if parts of the system prove to be undependable even in the face of a well-developed case, it will be important to analyze the case to understand which chain of evidence-argument-claim proved invalid.
Example of an Evolving Assurance Case
Expansion of a Reliability Claim. The expansion of the claim that "The Y subsystem satisfies its reliability requirements" could proceed as shown in Figure 5.
Figure 5: An Expansion of a Reliability Claim
Here we have chosen to show that subsystem Y is reliable by addressing possible hazards that might impair Y's reliability. The claims at the bottom of Figure 5 cover only a few of the hazards that might be listed. They might be suitable for the requirements phase in which details of the design are not available. The hazard analysis document at this phase might just be an extract from other requirements documents and as this document is revised, the assurance case will have to be updated as well.
Expansion Suitable for the Design Phase. For purposes of this example, we will expand the claim, "Y Timing Budgets are Satisfied." The expansion is shown in Figure 6.
Figure 6: Expansion Suitable for the Pre-Design Phase
The strategy is to appeal to analytical and testing results. The context element for the "Y Schedulability Analysis" claim indicates the type of analysis intended. The case at this stage is sufficiently developed considering that the actual or detailed design of the software has not yet been completed. The assurance plan at this stage would discuss the kinds of analyses being contemplated and who will produce the documents referenced in the context elements. Toward the end of the design phase, the case might look something like that shown in Figure 7.
Figure 7: Expansion After Design
This expands the context element for "Y Schedulability Analysis" to reference a document giving the detailed information required to conduct a schedulability analysis. The diagram also includes the results of the schedulability analyses, in "Ev: Y Analysis Results." (The circle is the shape used to describe evidence supporting claims.) The claim, "Y Testing," has been expanded to show that test results will be used for two purposes: to show that the schedulability model and analyses are consistent with the test results (this strengthens the validity of the schedulability analyses) and to show that test results indicate no violations of timing budgets.
We have also added a justification element (in the oval shape) indicating why the selected test cases will be useful in demonstrating that timing constraints are not violated. An explanation of when and how this justification will be created should appear in the updated assurance plan associated with this case.
Expansion for Integration and Evaluation Phase. Finally, toward the end of the test and evaluation phase, the assurance case would look something like Figure 8. The primary differences between this view and Figure 7 are the addition of evidence elements for the previously unexpanded claims and the adjustment of context and justification elements to reference the current versions of materials.
It is important to note that all developers of such a system, even in the absence of a requirement to present an assurance case, would already be doing extensive testing aimed at showing that the timing constraints were met. Most developers would also already be doing a formal schedulability analysis and would correlate the results as shown here. One difference is that the assurance case provides a structure for the already developed evidence that allows an outsider to more efficiently review the work product and makes it easier to convince them that the requirements have been met.
The amount of time spent in creating the diagrams summarizing the assurance case is small compared to the amount of time spent in generating the evidence and the detailed analyses serving as context or justification for claims. The effort to generate evidence is traditionally expended anyway; the assurance case simply provides a structure that helps integrate and clarify why the various forms of evidence support the claims.
Figure 8: Expansion After Implementation