|
Complete
Cleanroom software engineering is an engineering and managerial
process for the development of high-quality
software with certified reliability.
Cleanroom was originally developed by Dr. Harlan Mills [Linger
94, Mills
87]. The name "Cleanroom" was taken from the electronics
industry, where a physical clean room exists to prevent introduction
of defects during hardware fabrication. Cleanroom software
engineering reflects the same emphasis on
defect prevention rather than defect
removal, as well as certification of reliability for the intended
environment of use.
The focus of Cleanroom involves moving from traditional,
craft-based software development practices to rigorous,
engineering-based practices. Cleanroom software engineering yields
software that is correct by mathematically sound design, and software
that is certified by statistically-valid testing.
Reduced cycle time results from an
incremental development
strategy and the avoidance of rework.
It is well-documented that significant differences in cost are
associated with errors found at different stages of the
software life cycle. By detecting errors as
early as possible, Cleanroom reduces the cost of errors during
development and the incidence of failures during operation; thus the
overall life cycle cost of software developed under Cleanroom can be
expected to be far lower than industry average.
The following ideas form the foundation for Cleanroom-based
development:
- Incremental development under statistical
quality control (SQC). Incremental development as practiced in
Cleanroom provides a basis for statistical quality control of the
development process. Each increment is a complete iteration of the
process, and measures of performance in each increment (feedback)
are compared with preestablished standards to determine whether or
not the process is "in control." If quality standards are not met,
testing of the increment ceases and developers return to the
design stage.
- Software development based on mathematical principles. In
Cleanroom development, a key principle is that a computer program
is an expression of a mathematical function. The
Box Structure Method is used for specification and
design, and functional verification is used to confirm that the
design is a correct implementation of the specification.
Therefore, the specification must define that function before
design and functional verification can begin. Verification of
program correctness is performed through
team review based on correctness questions. There is no execution
of code prior to its submission for independent testing.
- Software testing based on statistical principles. In
Cleanroom, software testing is viewed as a statistical experiment.
A representative subset of all possible uses of the software is
generated, and performance of the subset is used as a basis for
conclusions about general operational performance. In other words,
a "sample" is used to draw conclusions about a "population." Under
a testing protocol that is faithful to the principles of applied
statistics, a scientifically valid statement can be made about the
expected operational performance of the software in terms of
reliability and confidence.
Benefits of Cleanroom include
significant improvements in correctness,
reliability,
and understandability.
These benefits usually translate into a reduction in
field-experienced product failures, reduced cycle time, ease of
maintenance, and longer product life.
Cleanroom has been documented to be very effective in new
development and reengineering (whole system or major subunits)
contexts. The following discussion highlights areas where Cleanroom
affects or differs from more conventional practice:
- Team-based development. A Cleanroom project team is small,
typically six to eight persons, and works in a disciplined fashion
to ensure the intellectual control of work in progress. Cleanroom
teamwork involves peer review of individual work, but does not
supplant individual creativity. Once the system architecture
has been established and the interfaces between subunits have been
defined, individuals typically work alone on a given system
component. Individual designs are working
drafts that are then reviewed by the team. In a large project,
multiple small teams may be formed, with one for the development
of each subsystem, thus enabling
concurrent engineering after
the top-level architecture has been established.
- Time allocation across life cycle phases. Because one of the
major objectives of Cleanroom is to prevent errors from occurring,
the amount of time spent in the design phase of a Cleanroom
development is likely to be greater than the amount of time
traditionally devoted to design. Cleanroom, however, is not a more
time-consuming development methodology,
but its greater emphasis on design and
verification often yields that concern.
Management understanding and acceptance of this essential point-
that quality will be achieved by design rather than through
testing- must be reflected in the development schedule. Design and
verification will require the greatest portion of the schedule.
Testing may begin later and be allocated less time than is
ordinarily the case. In large Cleanroom projects, where historical
data has enabled comparison of traditional and Cleanroom
development schedules, the Cleanroom schedule has equaled or
improved upon the usual development time.
- Existing organizational practices. Cleanroom does not preclude
using other software engineering techniques as long as they are
not incompatible with Cleanroom principles. Implementation of the
Cleanroom method can take place in a gradual manner. A
pilot project can provide an
opportunity to "tune" Cleanroom practices to the local culture,
and the new practices can be introduced as pilot results to build
confidence among software staff.
Initial Cleanroom use within IBM occurred in the mid to late 80s,
and project use continues to this day. Defense demonstration projects
began approximately 1992. Cleanroom has been used on a variety of
commercial and defense projects for which reliability was critically
important. Some representative examples include the following:
- IBM COBOL/SF product, which has required only a small fraction
of its maintenance budget during its operating history
[Hausler
94].
- Ericsson OS-32 operating system project, which had a 70%
improvement in development productivity, a 100% improvement in
testing productivity, and a testing error rate of 1.0 errors per
KLOC (represents all errors found in all testing) [Hausler
94].
- USAF Space Command and Control Architectural Infrastructure
(SCAI) STARS 1
Demonstration Project at Peterson Air Force Base in Colorado
Springs, CO. In this project, Cleanroom was combined with the TRW
(spiral) Ada Process Model and some generated and reused code to
produce software at a rate of $30-40 per line of code versus
industry averages of $130 per line for software of similar
complexity and development timeframe (the size of the application
is greater than 300 KLOC) [STARSSCAI
95].
- US Army Cleanroom project in the Tank-automotive and Armaments
Command at the U.S. Army Picatinny Arsenal. After seven project
increments (approximately 90K lines of code), a 4.2:1 productivity
increase and a 20:1 return on investment has been documented
[Sherer
96a, Sherer
96b]. This project experienced an overall testing error
rate (represents all errors found in all testing) of 0.5
errors/KLOC.
In 1995-1996, tools supporting various aspects of the Cleanroom
process became commercially available.
Using Cleanroom to accomplish piecemeal, isolated changes to a
system not developed using Cleanroom is not considered an effective
use of this technology. Training is required and commercially
available. Available courses range from overviews to a detailed focus
on particular aspects of Cleanroom. For some training classes, it is
most productive if software managers and technical staff take the
training together. Managers need a thorough understanding of
Cleanroom imperatives, and a core group of practitioners needs
sufficient orientation in Cleanroom practices to be able to adapt the
process to the local environment (this includes establishing a local
design language, local verification standards, etc.).
Cleanroom and
object-oriented methods. A
study/analysis of Cleanroom and three major object-oriented methods:
Booch, Objectory, and Shlaer-Mellor (see
Object-Oriented Analysis), found that combining object-oriented
methods (known for their focus on reusability) with Cleanroom (with
its emphasis on rigor, formalisms, and reliability) can define a
process capable of producing results that are not only reusable, but
also predictable and of high quality. Thus object-oriented methods can
be used for front-end domain analysis
and Cleanroom can be used for life-cycle application engineering
[Ett
96].
Cleanroom and the Capability Maturity
Model (CMM). A cleanroom Reference Model [Linger
96b] (a set of Cleanroom Processes for software management,
specification, design, and certification) and a detailed mapping of
Cleanroom to the CMM for Software [Linger
96a] have been created. The mapping shows that Cleanroom and
the CMM [Paulk
93] are fully compatible and mutually reinforcing.
This technology is classified under the following categories.
Select a category for a list of related topics.
|
[Cleanroom 96]
|
Cleanroom Tutorial
[online]. Available WWW
<URL:
http://source.asset.com/stars/loral/cleanroom/tutorial/cleanroom.html>
(1996).
|
|
[Ett 96]
|
Ett, William & Trammell, Carmen. A
Guide to Integration of Object-Oriented Methods and
Cleanroom Software Engineering [online].
Originally available WWW
<URL:
http://www.asset.com/stars/loral/cleanroom/oo/guidhome.htm>
(1996).
|
|
[Hausler 94]
|
Hausler, P. A.; Linger, R. C.; &
Trammel, C. J. "Adopting Cleanroom Software Engineering with
a Phased Approach." IBM Systems Journal 33, 1 (1994):
89-109.
|
|
[Linger 94]
|
Linger, R.C. "Cleanroom Process Model."
IEEE Software 11, 2 (March 1994): 50-58.
|
|
[Linger 96a]
|
Linger, R.C.; Paulk, M.C.; & Trammel,
C.J. Cleanroom Software Engineering Implementation of the
CMM for Software (CMU/SEI-96-TR-023). Pittsburgh, PA:
Software Engineering Institute, Carnegie Mellon University,
1996.
|
|
[Linger 96b]
|
Linger, R.C. & Trammel, C.J.
Cleanroom Software Engineering Reference Model
(CMU/SEI-96-TR-022). Pittsburgh, PA: Carnegie Mellon
University, Software Engineering Institute, 1996.
|
|
[Mills 87]
|
Mills, H.; Dyer, M.; & Linger, R.
"Cleanroom Software Engineering." IEEE Software 4, 5
(September 1987): 19-25.
|
|
[Paulk 93]
|
Paulk, M.; Curtis B.; Chrissis, M.; &
Weber, C. Capability Maturity Model for Software
Version 1.1 (CMU/SEI-96-TR-24, ADA263403). Pittsburgh, PA:
Software Engineering Institute, Carnegie Mellon University,
1993.
|
|
[Sherer 96a]
|
Sherer, S. W. Cleanroom Software
Engineering- the Picatinny Experience [online].
Available WWW
<URL:
http://software.pica.army.mil/cleanroom/cseweb.html>
(1996).
|
|
[Sherer 96b]
|
Sherer, S.W.; Kouchakdjian, A.; &
Arnold, P.G. "Experience Using Cleanroom Software
Engineering." IEEE Software 13, 3 (May 1996):
69-76.
|
|
[STARSSCAI 95]
|
Air Force/STARS Demonstration Project
Home Page [online]. Available WWW
<URL:
http://www.asset.com/stars/afdemo/home.html>
(1995).
|
John Foreman, SEI
Wayne Sherer, US Army Picatinny Arsenal
Dave Ceely, Lockheed Martin, Gaithersburg, MD
Dr. Jesse Poore, President, Software Engineering Technologies
(SET)
Rick Linger, SEI
27 Oct 97: updated URL for [Ett 96]
20 Jun 97: updated URL for [Ett 96]
10 Jan 97: original
1 STARS: Software Technology for
Adaptable Reliable Systems
The Software
Engineering Institute (SEI) is a federally funded research and
development center sponsored by the U.S. Department of Defense
and operated by Carnegie Mellon University.
Copyright
2007
by Carnegie Mellon University
Terms of Use
URL: http://www.sei.cmu.edu/str/descriptions/cleanroom_body.html
Last Modified: 11 January 2007
|
