Clarifying the CMMI for Development Model for High Maturity

NEWS AT SEI

Author

Mike Phillips

This library item is related to the following area(s) of work:

Process Improvement
CMMI

This article was originally published in News at SEI on: July 1, 2008

The CMMI Steering Group and the SEI have agreed that the material contained in Version 1.2 of the CMMI for Development (CMMI-DEV) model should be clarified to better explain high maturity. These clarifications will be designed to address the needs that CMMI users, appraisers, and instructors have expressed related to CMMI high maturity practices. This column describes the reasons and current planning for how high maturity will be clarified.

What will be clarified?

Part of the purpose of the Version 1.2 release was to strengthen the integrity of CMMI appraisal results. An important aspect of this purpose included high maturity and capability (i.e., levels 4 and 5). Actions taken as part of Version 1.2 included the following:

  • A certification process for high maturity lead appraisers  was established.
  • An appraisal lifetime policy was established,  which limited the viability of appraisal results to three years.
  • Rigorous reviews of appraisal results were  defined and required to enable public announcements of appraisal rating results.
  • Resources were committed to allow audits of all appraisals targeting level 4 and 5 appraisal ratings for several months. These audits have helped us assure that the fundamentals for CMMI high maturity are in evidence in the appraised organizations claiming to achieve these levels. More information about the policy regarding high maturity appraisal audits can be found at www.sei.cmu.edu/appraisal-program/appraiser-communications/index.html.

The SEI hired two experienced lead appraisers to augment its quality assurance efforts early in 2008 and began conducting dedicated high maturity audits in February. As of October 2008, the SEI has conducted 29 audits of high maturity appraisals. These audits have been conducted both onsite and remotely. The time differences involved in some appraisals conducted half-way round the world from the audit team adds to the challenges faced by both the appraisal team and the audit team.

Remote audits involve an audit team that requests and receives appraisal information on high maturity elements from the appraisal team as the appraisal is being conducted. These high maturity elements are strong indicators that the organizations (and the high maturity lead appraisers) fully understand the fundamentals of high maturity and are properly using the tools needed to achieve high maturity and high capability (i.e., statistical management and process improvement models).

Audits are difficult to conduct, but are worth the trouble because of the valuable information collected. The valuable information received from those audited has confirmed that these audits serve a useful purpose of maintaining the integrity of appraisal results. We will soon shift our focus from auditing all high maturity appraisals to a sampling approach. This approach will assure that we do not focus exclusively on high maturity. There is a much greater number of appraisals aimed at achieving maturity levels 2 or 3 and others that focus on achieving capability profiles, without seeking maturity level equivalence.

Usually, audits confirm the findings of the appraisal team. Such audit results not only confirm the appraisal results for the organization, but also confirm the process and judgment used by the lead appraiser and his or her team.

Occasionally, audits uncover appraisal deficiencies. Examples of deficiencies include errors in judgment by high maturity lead appraisers and the evidence provided by the organization being appraised. In such instances, audit teams work closely with lead appraiser to ensure that the appraisal is brought to an appropriate conclusion. The outcomes of audits in these situations have resulted in the decertification of some high maturity lead appraisers, and in some cases rejection of submitted appraisal results.

So what is the plan?

The CMMI Steering Group and the SEI have agreed that they want to provide more help for users targeting level 4 and 5 appraisal results. Although there are white papers now available on the SEI website that provide clarification of high maturity practices, all agree that more is needed.

The first initiative is to create a set of criteria that describes the particular elements of the appraisal that are being audited. We think that communication of these criteria can help alleviate fear of the unknown that all audits can cause. The initial criteria will be identified with leadership from industry and approved by the Steering Group to assure that the breadth of model interest and experience is represented. As with other CMMI elements, the SEI will provide a change request mechanism so that further improvements can be made.

Work is also being done to craft improvements to the text of the CMMI-DEV model. These improvements are being developed in the form of model redlines. These redlines were reviewed by the high maturity lead appraisers and the CMMI Steering Group at a workshop in late September. As a result of the review, the Steering Group determined that modernizing the practices for levels 4 and 5 was a better choice than a more limited attempt to clarify the information. These improvements will be incorporated into the planned release of CMMI, Version 1.3 for all three CMMI constellations rather than in a CMMI-DEV, Version 1.2a model.The Steering Group and the SEI also recognize that suggested improvements to high maturity practices must come from a wider community than only the appraisers. Therefore, more opportunities like the September workshop are being considered. Where possible, we will link with events such as the November CMMI Conference in Denver and the March SEPG Conference in San Jose by providing opportunities to contribute ideas in special sessions held at these conferences.

Also planned are improvements and clarifications to the auditing processes now conducted by the SEI to evaluate high maturity appraisals. If an audit results in decertification, an appeal process has been defined that will provide a way for the lead appraiser to request a review of the judgment made by the CMMI Appraisal Program. Documentation of the appeal process is being distributed to high maturity lead appraisers for their use. This appeal process includes a review of the contested appraisal by a board of lead appraisers, with the voting members from outside the SEI.

If the result of an audit is that the appraisal results are rejected by the SEI, a process is available to the organization to contest this outcome. For organizations concerned about appraisal results, an adjudication process is being made available to assure the organization’s understanding of the reasons for the SEI’s determination. Further, the process allows a review by high maturity lead appraisers external to the SEI to provide an independent cross-check of the SEI quality assurance determination. Detailed descriptions of these processes are found on the SEI's Appraiser Program pages. www.sei.cmu.edu/appraisal-program/appraiser-communications/index.html.

Summary

High maturity thinking has come a long way since the levels associated with the term were created in the early 1990s. The effective use of statistical techniques to manage and improve processes has advanced with the continuing evolution of tools that assist in measuring and predicting process performance. The CMMI Steering Group and SEI believe that the steps described in this column will help ensure that CMMI users more clearly understand and interpret CMMI practices and achieve their process improvement goals.

About the Author

   

As the director of special projects at the Software Engineering       Institute, Mike Phillips leads the Capability Maturity Model Integration (CMMI) project for the SEI. He was previously responsible for transition-enabling       activities at the SEI. Prior to his retirement as a colonel from the Air       Force, he managed the $36B development program for the B-2 in the B-2 SPO       and commanded the 4950th Test Wing at Wright-Patterson AFB, Ohio. In addition       to his bachelor’s degree in astronautical engineering from the U.S.       Air Force Academy, Phillips has master’s degrees in nuclear engineering       from Georgia Tech, in systems management from the University of Southern       California, and in international affairs from Salve Regina College and the       Naval War College.

The         views expressed in         this article are the         author's only and do         not represent directly         or imply any official         position or view of         the Software Engineering         Institute or Carnegie         Mellon University.         This article is intended         to stimulate further         discussion about this         topic.

Please note that current and future CMMI research, training, and information has been transitioned to the CMMI Institute, a wholly-owned subsidiary of Carnegie Mellon University.

Find Us Here

Find us on Youtube  Find us on LinkedIn  Find us on twitter  Find us on Facebook

Share This Page

Share on Facebook  Send to your Twitter page  Save to del.ico.us  Save to LinkedIn  Digg this  Stumble this page.  Add to Technorati favorites  Save this page on your Google Home Page 

For more information

Contact Us

info@sei.cmu.edu

412-268-5800

Help us improve

Visitor feedback helps us continually improve our site.

Please tell us what you
think with this short
(< 5 minute) survey.