Using CMMI: How Is it Progressing?

NEWS AT SEI

Author

Mike Phillips

This library item is related to the following area(s) of work:

CMMI
Process Improvement

This article was originally published in News at SEI on: September 1, 2003

A central purpose of this column is to keep you updated on the adoption progress we are seeing as organizations upgrade to CMMI. In this issue, I’ll summarize both what we have seen and what we see coming in the near future.

Adoption Indicators

One of the difficulties of characterizing the adoption of any SEI technology is to determine what measures to use. Anecdotal information is commonly used and easy to obtain. For example, one large aerospace company has banners declaring “CMMI Level 3 in ’03.” But such anecdotes could lead to the incorrect conclusion that only specific types of organizations are upgrading to CMMI.

The two continuing sources of quality data we have come from our transition partners--qualified DoD and industry organizations authorized by the SEI to help other organizations adopt new and improved technologies--and from SEI staff members. Both of these groups provide training and appraisal services to organizations all around the world and have seen first hand the results of CMMI use.

I find the most telling measure to be the number of people who have taken the “Introduction to CMMI” training course. As of the end of July of this year, a total of 8,837 people have taken the three-day course since its inception a year and a half earlier. This compares with just over 18,000 who have taken the “Introduction to CMM” course, which has been taught for over ten years and will have its last public offering at the SEI in December. We are pleased with this healthy start in CMMI course attendance. About 80% of CMMI instruction has been provided by our transition partners, whose ability to deliver their services at a myriad of sites has had significant impact on CMMI adoption.

The other indicator provided to the SEI is information about appraisals. The number of SCAMPI appraisals is also growing nicely, but not at the same rate as course attendance. As of the end of July, the number of reported Class A SCAMPIs was 133. Last year SCAMPIs represented 13% of the total SEI-supported benchmark appraisals reported. So far this year, the percentage has risen to 22%. I believe this may slightly understate the pace of transition, since we have de-emphasized the use of benchmark-scale Class A appraisals early in the process improvement life cycle. Anecdotal reports of various Class B and C appraisals for initial gap analysis and progress measurement suggest that the adoption curve is healthy.

We have also begun to examine the appraisal data collected to date. While the information thus far is based on hundreds rather than thousands of appraisals, the data confirms what we had hoped. First, the upgrade to CMMI is not a huge hurdle that forces organizations back to a lower maturity level. Capability Maturity Model for Software (SW-CMM) users who have achieved a maturity level and then adopt CMMI typically are able to achieve a CMMI maturity level that matches or exceeds their SW-CMM maturity-level rating.

Second, the process improvement benefits experienced by those who adopt CMMI typically exceed, and do not fall below, benefits experienced by SW-CMM users who achieve the same maturity level.

The amount of data collected thus far is too small to make any similar assertions about the effect of using CMMI models that include more disciplines, such as integrated product and process development (IPPD) and/or supplier sourcing. We hope to see evidence that each of these expansions offers progressively better integration and improved quality.

Some of you have been interested in which of the two representations is used more, staged or continuous. Data concerning this issue is ambiguous. Those visiting the CMMI Web site select the continuous representation more often than staged. This pattern suggests that users download the continuous representation for use. However, the vast majority of appraisals reported to the SEI have used the staged representation. Most likely this is because the staged approach gives a direct measure of the familiar maturity level. (While a maturity level is also achieved by using the continuous representation with equivalent staging, few lead appraisers have reported their findings that way.) Course attendance is mixed, with a 60/40 split between staged and continuous. I suspect that the number of course offerings for each representation may cause the only differences, as attendance seems equally strong in each course offering.

Finally, we’ve found that the CMMI Web site has had 800,000 to 900,000 hits every month for the past six months. Those hits are originating from countries all over the world--Russia, Indonesia, Latvia, Argentina, Chile, Italy, Thailand, Israel, and many others--and from a mix of government, commercial, and academic organizations. One of the most highly accessed Web pages is the page that provides the Word version of CMMI-SW. Its popularity may result from users viewing it as the most relevant first step for those upgrading from the SW-CMM.

CMMI Product Suite Updates

When we launched the CMMI V1.1 Product Suite in early 2001, we committed to maintaining the stability of the current version for at least three years. We maintain our commitment to that approach. We receive occasional change requests and have regularly updated an errata sheet for each model that corrects errors found since the model release. There have also been requests to expand CMMI best practices to cover areas such as safety and security assurance and hardware engineering. Other requests include expanding best practices to cover more of the product life cycle, including manufacturing, operations, and disposal.

To begin considering when and which upgrades should be made to the CMMI Product Suite, we announced in September a 90-day comment period, open to current users of the product suite. Information received from those using the model today will help guide the plans for corrections and improvements tomorrow. Besides the input received during the 90-day comment period, we will also consider change requests submitted since release of V1.1 and input received as part of the CMMI interpretive guidance project.

If you submitted change requests for CMMI V1.0 or V1.02 and your change requests were not incorporated, you must resubmit your comments using the public-review process if you continue to have the same concerns.

The drafting and piloting of proposed changes will take us well into 2005 at the soonest. We believe that the scope of change for the next update will likely be minor, in which case a version V1.2 of the model and appraisal method will result. Even if the scope of changes are extensive enough to result in a V2.0, we are committed to ensuring that these changes will align with the existing suite so that no repetition of training and deployment, such as those associated with V1.1 and V1.0, will be necessary.

Work in Progress

Initial Coverage for Acquisition

One of the disciplines considered for eventual inclusion in the CMMI Framework is acquisition. In December 2000, draft process areas in CMMI V1.02d were released as a way of exploring the possibility of incorporating this discipline. This draft later evolved into the supplier sourcing (SS) addition to V1.1. The treatment of supplier sourcing consisted of adding a single process area, Integrated Supplier Management, and informative amplifications of various practices in a few of the existing process areas. This addition of a discipline increased attention on analyzing and selecting suppliers and improving customer–supplier interactions. In spite of this step forward for outsourcing, we determined that further acquisition practices would be needed in the future to aid the government’s acquisition efforts.

The CMMI Steering Group has recently asked a group to create a document that will provide broader coverage of acquisition within the CMMI Framework and meet the needs of government acquirers. This approach will provide opportunities for piloting the best practices in the document over the next year, so that a future upgrade to the CMMI Product Suite can be considered. As I’ve mentioned before, this approach allows us to maintain the stability of the V1.1 Product Suite but expand the communities able to benefit from CMMI.

Technical Notes

It is now easier than ever to find all CMMI-related reports and technical notes on the SEI Web site. Because of the importance of these reports and technical notes for building the understanding of CMMI-related information, we have created a CMMI Web page for CMMI-related reports.

In the “coming soon” category, we have a few technical notes in various stages of preparation. One in final editing that may be available very shortly was produced by an organization that provides engineering services as a major part of its business. The challenges of developing the services, operating in a dynamic customer environment, and modifying services quickly all make the use of CMMI practices beneficial, although the existing practices often require some interpretation to more closely fit the environment of service providers.

Another team is working to characterize the best safety and security engineering practices to improve the development of safety-critical and security-critical systems. The team has proposed an approach in which there are two goals in a single process area and amplifications for these two disciplines that can be added to the informative material in various parts of existing process areas without changing the goals or practices in those process areas.

Interpretive Guidance

The interpretive guidance project was formed to collect information about how CMMI is being used by software, information technology (IT), and information systems (IS) organizations. Information was collected through a variety of channels and sources in the software and IT/IS communities. An upcoming preliminary report will reveal the data-collection methods used and the raw data collected. The report will also provide some early insight into the adoption of CMMI by the organizations that chose to participate.

Some highlights of this report include the following:

  • When asked “Has your organization made a decisionabout adopting CMMI?” 48% of the respondents stated that adoption was in progress, 15% stated that CMMI was well institutionalized throughout their organizations, 10% stated that their organizations chose not to adopt CMMI, and 23% said the decision to adopt had not yet been made.
  • When asked if, in their opinion, CMMI is adequate for guiding process improvement, over 77% of respondents agreed or strongly agreed.
  • When asked if including both systems engineering and software in a single model has been a help for them, nearly 65% agreed or strongly agreed with the statement.

Refer to the report titled CMMI Interpretive Guidance Project Preliminary Report, for more information.

About the Author

Mike Phillips is the Director of Special Projects at the SEI, a position created to lead the Capability Maturity Model Integration (CMMI) project for the SEI. He was previously responsible for transition-enabling activities at the SEI.

Prior to his retirement as a colonel from the Air Force, he managed the $36B development program for the B-2 in the B-2 SPO and commanded the 4950th Test Wing at Wright-Patterson AFB, OH. In addition to his bachelor’s degree in astronautical engineering from the Air Force Academy, Phillips has masters degrees in nuclear engineering from Georgia Tech, in systems management from the University of Southern California, and in international affairs from Salve Regina College and the Naval War College.

Please note that current and future CMMI research, training, and information has been transitioned to the CMMI Institute, a wholly-owned subsidiary of Carnegie Mellon University.

Find Us Here

Find us on Youtube  Find us on LinkedIn  Find us on twitter  Find us on Facebook

Share This Page

Share on Facebook  Send to your Twitter page  Save to del.ico.us  Save to LinkedIn  Digg this  Stumble this page.  Add to Technorati favorites  Save this page on your Google Home Page 

For more information

Contact Us

info@sei.cmu.edu

412-268-5800

Help us improve

Visitor feedback helps us continually improve our site.

Please tell us what you
think with this short
(< 5 minute) survey.