Measurement and Analysis: What Can and Does Go Wrong?
By now, one can point to many instances where
measurement has been used effectively to inform
management and technical decisions in support of the
development and maintenance of software and
software intensive systems. Yet, measurement is not
very well integrated into software or systems
engineering education or practice, and measurement
remains challenging for all too many organizations.
For this paper, we have analyzed 1350 findings drawn
from 663 Software CMM appraisals that were
conducted between 1987 and 2002 inclusive. The
results are augmented by questions from a survey of
CIO’s from state and local governments and the
private sector. Our analyses suggest several areas
where both managers and engineers would benefit
from better guidance about the proper use of
measurement and analysis. Future work may include
lexical analyses based on natural language processing
as well as studies of appraiser understanding of the
measurement content in CMMI models.