|
Complete
Software Inspections are a disciplined engineering practice for
detecting and correcting defects in
software artifacts, and preventing their leakage
into field operations. Software Inspections were introduced in the
1970s at IBM, which pioneered their early adoption and later
evolution [Fagan
76]. Software Inspections provide value in improving software
reliability, availability, and maintainability.
Many organizations have made commitments to initiatives in the
Capability Maturity Model® (CMM®)1,
ISO 9000, or Six Sigma in order to deliver superior quality. Each of
these initiatives has one thing in common: the practice of Software
Inspections.
Experienced software practitioners and managers understand that
software development is a process of experimentation involving the
continuous discovery of technical information associated with the
function, form, and fit of the software product. Software Inspections
are an integral practice in the process of experimentation.
Software inspections provide value in improving
reliability,
availability, and
maintainability.
Software Inspections are strict and close examinations conducted
on requirements, specifications, architectures, designs, code, test
plans and procedures, and other artifacts [Ebenau
94], [O'Neill
01a]. Leading software indicators of excellence for each
artifact type provide the exit criteria for the activities of the
software life cycle. For example, these indicators include
completeness, correctness, style, rules of construction, and multiple
views [O'Neill
88,92].
Completeness is based on traceability of the requirements to the
code, essential for maintainability. Correctness is based on the
clear specification of intended function and its faithful elaboration
in code, essential for reliability and availability [Linger
79]. Style is based on consistency of recording, essential
for maintainability. Rules of construction are based on the software
application architecture and the specific protocols, templates, and
conventions used to carry it out, essential for reliability and
availability. Multiple views are based on the various perspectives
and viewpoints required to be reflected in the software product,
essential for maintainability. By detecting defects early and
preventing their leakage into subsequent activities, the need for
later detection and rework (which is essential for reduced cycle time
and lower cost) is eliminated.
Software Inspections are a reasoning activity performed by
practitioners playing the defined roles of moderator, recorder,
reviewer, reader, and producer. Each role carries with it the
specific behaviors, skills, and knowledge needed to achieve the
expert practice of Software Inspections [Freedman
90].
The adoption of Software Inspections practice is competency
enhancing and meets little resistance among practitioners trained in
their use. The adopting organization benefits by improved
predictability in cost and schedule performance, reduced cost of
development and maintenance, reduced defects in the field, increased
customer satisfaction, and improved morale among practitioners.
The Return on Investment for Software Inspections is defined as
net savings divided by detection cost [O'Neill
01a,c].
Savings result from early detection and correction avoiding the
increased cost that comes with the detection and correction of
defects later in the life cycle. An undetected major defect that
escapes detection and leaks to the next phase may cost two to ten
times to detect and correct [Basili/Boehm
01]. A minor defect may cost two to four times to detect and
correct. The net savings then are up to nine times for major defects
and up to three times for minor defects. The detection cost is the
cost of preparation effort and the cost of conduct effort.
While Software Inspections originated and evolved in new
development, its usefulness in maintenance is now well established.
Certain measurements obtained during Software Inspections reflect
this context of use. For example, the lines of code inspected per
conduct hour range from 250 to 500 for new development and from 1000
to 1500 for maintenance. Other measurements reveal no distinction
between these contexts of usage. For example, the defects detected
per session range from five to ten for both new development and
maintenance.
The organization adopting Software Inspections practice seeks to
prevent defect leakage from one life cycle activity to another.
Following training, the organization can expect to detect 50% of the
defects present. It may take 12 to 18 months to achieve expert
practice where defect detection is expected to range from 60% to 90%
[O'Neill
89], [O'Neill
01a].
The maturity of a technology can be reasoned about in terms of its
long-term, widespread use in a variety of usage domains and its
transition from early adopters through late adopters. Software
Inspections have been evolving for 25 years. They are known to
deliver economic value.
The data discussed in Usage Considerations above and Costs and
Limitations below are drawn from the National Software Quality
Experiment (NSQE) [O'Neill
95,96,00] where thousands of participants from dozens of
organizations are populating the experiment database with thousands
of defects of all types along with pertinent information needed to
pinpoint their root causes. The analysis bins identified in the
experiment include software process maturity level (1,2,3...),
organization type (government, Department of Defense (DoD) industry,
commercial), product type (embedded, organic), programming language
(old style, modern), and global region (North America, Pacific Rim,
Latin America).
Organizations are invited to calibrate their Software Inspection
results with the NSQE results using the Software Inspection
Measurement and Derived Metrics tool [O'Neill
01b] found at http://members.aol.com/ONeillDon/nsqe-assessment.html.
Software Inspections are a rigorous form of
peer reviews, a Key Process Area (kpa) of
the CMM [Paulk
95], [Humphrey
89]. Although peer reviews are part of achieving CMM level 3,
and many organizations limit their software process improvement
agenda to the kpas for the maturity level they are seeking to
achieve, the population of Software Inspections adopters ranges from
level 1 to 5.
The rollout and operating costs associated with Software
Inspections include the initial training of practitioners and
managers, the ongoing preparation and conduct of inspection sessions,
and the ongoing management and use of measurement data for defect
prevention and return on investment computations.
To properly adopt Software Inspections practice, each participant
is trained in the structured review process, defined roles of
participants, system of process and product checklists, and forms and
reports. The lost opportunity cost to acquire the knowledge, skills,
and behaviors is twelve hours per practitioner [O'Neill
89]. In addition, each manager is trained in the
responsibilities for rolling out the technology and the
interpretation and use of measurements taken. The management training
is accomplished in four hours.
The cost of performing Software Inspections includes the
individual preparation effort of each participant before the session
and the conduct effort of participants in the inspections session.
Typically, 4-5 people participate and expend 1-2 hours of preparation
and 1-2 hours of conduct each. This cost of 10 to 20 hours of total
effort per session results in the early detection of 5-10 defects in
250-500 lines of new development code or 1000-1500 lines of legacy
code [O'Neill
95,96,00].
The National Software Quality Experiment (NSQE) [O'Neill
95,96,00] reveals that the Return on Investment (net
savings/detection cost) for Software Inspections ranges from four to
eight independent of the context of usage. Organizations are invited
to calibrate their software inspection return on investment using the
tool [O'Neill
01c] found at http://members.aol.com/ONeillDon/nsqe-roi.html.
In order for Software Inspections to be systematically used in
statistical process control, there must be a life cycle model with
defined software artifacts. In this context, Software Inspections
provide the exit criteria for each life cycle activity. Furthermore,
the standard of excellence of leading indicators for each type of
artifact must be specified and used in practice.
While Software Inspections are a rigorous form of peer reviews,
software walkthroughs are a less rigorous
form of peer reviews [O'Neill
01a]. Walkthroughs may cost as much as inspections, but they
deliver less. Notably, walkthroughs provide no measured results and
that precludes the application of statistical process control needed
to advance software process maturity.
To optimize the practice of Software Inspections on legacy code
during maintenance operations, all modules are rank ordered by
cyclomatic complexity.
Candidates for inspection are selected from those with highest
complexity rating where the defect density is expected to be
high.
This legacy code maintenance strategy can be extended by rank
ordering all modules based upon incidents encountered in the past
year and by rank ordering the modules expected to be adapted and
perfected in the coming year. Modules for inspection are then
selected based on their rank ordering in cyclomatic complexity,
defect history, and expected rework.
This technology is classified under the following categories.
Select a category for a list of related topics.
|
[Basili/Boehm 01]
|
Basili, Vic, & Barry Boehm. "Software
Defect Reduction Top 10 List." Computer 34,1,
(January 2001): 135-137.
|
|
[Ebenau 94]
|
Ebenau, Robert G. & Strauss, Susan H.
Software Inspection Process. New York, NY:
McGraw-Hill, 1994.
|
|
[Fagan 76]
|
Fagan, M. "Design and Code Inspections to
Reduce Errors in Program Development." IBM Systems
Journal 15, 3 (1976): 182-211.
|
|
[Freedman 90]
|
Freedman, D.P. & Weinberg, G.M.
Handbook of Walkthroughs, Inspections, and Technical
Reviews. New York, NY: Dorset House, 1990.
|
|
[Gilb 93]
|
Gilb, Tom & Graham, Dorothy.
Software Inspection. Essex, England: Addison-Wesley
Longman Ltd., 1993.
|
|
[Humphrey 89]
|
Humphrey, Watts S. Managing the Software
Process. Reading, MA: Addison-Wesley, 1989.
|
|
[Linger 79]
|
Linger, R.C.; Mills, H.D.; & Witt,
B.I. Structured Programming: Theory and Practice. Reading,
MA: Addison-Wesley, 1979.
|
|
[O'Neill 88]
|
O'Neill, Don & Ingram, Albert L.
"Software Inspections Tutorial," 92-120. Software
Engineering Institute Technical Review 1988.
Pittsburgh, PA: Carnegie Mellon University, Software
Engineering Institute, 1988.
|
|
[O'Neill 89]
|
O'Neill, Don. Software Inspections
Course and Lab. Pittsburgh, PA: Software Engineering
Institute, Carnegie Mellon University, 1989.
|
|
[O'Neill 92]
|
O'Neill, Don. "Software Inspections: More
Than a Hunt for Errors." Crosstalk, Journal Of Defense
Software Engineering 30 (January 1992): 8-10.
|
|
[O'Neill
95,96,00]
|
O'Neill, Don. "National Software Quality
Experiment: Results 1992-1999." Software Technology
Conference, Salt Lake City, 1995, 1996, and 2000.
|
|
[O'Neill 01a]
|
O'Neill, Don. Peer Reviews.
Encyclopedia of Software Engineering. New York, New York:
Wiley Publishing, Inc., to appear 2001.
|
|
[O'Neill 01b]
|
O'Neill, Don. Software Inspection
Measurements and Derived Metrics Tool [online].
Available WWW <URL: http://members.aol.com/ONeillDon/nsqe-assessment.html>
(2001).
|
|
[O'Neill 01c]
|
O'Neill, Don. Return on Investment Tool
[online]. Available WWW <URL: http://members.aol.com/ONeillDon/nsqe-roi.html>
(2001).
|
|
[Paulk 95]
|
Paulk, Mark C. The Capability
Maturity Model: Guidelines for Improving the Software
Process. Reading, MA: Addison-Wesley Publishing
Company, 1995.
|
Don O'Neill, Don O' Neill Consulting
Alex Elentukh, Fidelity Investments
Rick Linger, SEI
Watts Humphrey, SEI
Joan Weszka, Lockheed Martin
10 Jan 97 (original)
1 April 01 (updated)
1 Capability Maturity Model and CMM are
service marks of Carnegie Mellon University.
The Software
Engineering Institute (SEI) is a federally funded research and
development center sponsored by the U.S. Department of Defense
and operated by Carnegie Mellon University.
Copyright
2007
by Carnegie Mellon University
Terms of Use
URL: http://www.sei.cmu.edu/str/descriptions/inspections_body.html
Last Modified: 11 January 2007
|
