NEWS AT SEI
This article was originally published in News at SEI on: December 1, 1999
In this article, SEI staff and members of the CMMI Steering Group engage in a wide-ranging discussion about the CMMI project. The participants are:
Sandy Shrum: What would you say are the advantages of using a CMMI model over other process-improvement models?
Mike Phillips: The key letter in CMMI is the "I." The whole reason we are on this journey is to create a better improvement tool that integrates a number of important disciplines and ways of doing business across the enterprise. We think there is a particular advantage to bringing complementary disciplines into one fold and making sure that process improvement benefits the entire enterprise. Specifically, in the initial phase, we are bringing the engineering disciplines together.
Bob Rassa: The CMMI process-improvement methodology will help eliminate stovepipe environments and redundancy in a company’s process-improvement environment. When you’re dealing with separate stovepipe models, you tend to build processes around those. That tends to cause redundancy because you get a systems engineering group that will build their systems engineering processes and a software group that builds their software processes. There is also the issue of a sense of methodology. The CMMI product suite will have a common assessment methodology that will help to simplify the assessment process.
Hal Wilson: The integration of software and systems engineering in our organization is the key to the value of the CMMI models. Having started the process of continuing the improvement of software as well as beginning the process of systems engineering process improvement, it became very clear to us that there were, as Bob indicated, some elements of stovepiping in that process and a lot of redundancy. We see the big value in the fact that the CMMI models bring the two together in a cohesive fashion and, in fact, are more cohesive than we expected they could be. That is a big advantage to us. Another advantage is the existence of a more complete software product engineering model to match up with the elements of systems engineering. That is a tremendous advantage and improvement over the previous software-engineering-only model.
Dennis Ahern: I support everything my colleagues have said. Where I work [at Northrop Grumman], we began to develop an integrated engineering process several years ago. Actually we call it "Integrated Enterprise Process," and it includes software and systems engineering plus a number of other engineering disciplines. We recognized this need long ago, and I think we were not alone. In talking with colleagues at other companies, I know that other organizations have started down this path. Indeed, the Federal Aviation Administration recognized a similar need years ago. I think that integrated process improvement clearly represents something in which industry has demonstrated an interest and wishes to see occur.
Joan Weszka: I’d like to think that the use of the CMMI model has some of the same advantages as using the Integrated Product and Process Development (IPPD) method. Specifically, we’re promoting timely collaboration of all the disciplines throughout the life cycle--not just the product development life cycle, but also the process-improvement life cycle, as defined by the SEI IDEAL model. As a result of discipline collaboration, we’d expect that there would be a yield of both quality improvement and cost savings. Like Northrop Grumman, Lockheed Martin has also integrated its processes across the corporation. We have integrated them from both a software and systems engineering perspective and from an IPPD perspective. What we would expect to see as an advantage to using the CMMI model is the ability to adapt an industry standard to our integration paradigm.
Jack Ferguson: CMMI allows a much broader process improvement rather than being stovepiped into little communities. Also, especially for the software people, it provides more emphasis on software engineering versus the management emphasis in the software model. As we talk to software engineers about the CMMI approach, we discuss the relatively modest expansion of software product engineering to the other process areas that we inherited from systems engineers. You see a lot of people’s heads nod when we say that we’ve turned many of these activities that were of fairly limited scope in the old software model into process areas and fleshed them out with more detail. The technical people on the software side said they appreciate that, and that’s where we should be headed. We talked to the process-improvement people who are active in Europe; and they said that they’ve been telling their customers for a long time that they need to worry about not only the management of process improvement, but the technical aspects of improving software development processes.
Shrum: Let’s compare the CMMI model to the existing models. If I’m with a company that is already using the Software CMM Version 1.1, how would you persuade me that I should be using the new integrated model?
Weszka: I think the biggest advantage is that the CMMI models provide a growth path for enterprise-wide process improvement. For example, if today you’re using the Software CMM for process improvement, in the future you might want to expand your scope--into systems engineering, into IPPD, or into other disciplines that will be included under CMMI models in the future. As these additional disciplines are added, the organization can expand its process-improvement efforts to maximize the benefit and the return on investment.
Ahern: Like anything, a Capability Maturity Model can be improved. Version 1.1 of the Software CMM has been used for some time. Before the CMMI effort began, it was already recognized that there were areas where the Software CMM could be improved. In fact, there were drafts of another release that we on the CMMI team used as one of our sources. We think that the CMMI model that we released for public review incorporates a lot of the improvements that the community using Version 1.1 had already recognized and wanted to see in the next release. We’re hoping that as the CMMI model gets reviewed, we will learn that we were successful in putting those improvements in, because I think we have been successful in large part.
Wilson: I would agree with both of those statements. I think they’re probably the key ones. As I mentioned before, we believe that the extension of the software product engineering activities within the software-only model to be more specific in the areas of validation and verification, and other areas have really made an improvement in the model. That’s another reason to say that, even if you’re just doing software engineering and just using Version 1.1, it would be worth moving to the CMMI software-only version.
Rassa: Recognize that Software CMM Version 2.0, which was not released, was also going to raise the bar for software process improvement. The new CMMI model used as its baseline document SW-CMM Version 2.0, so the new model raises the bar and that may deter some organizations from going that way, from adopting the CMMI model. They’re going to say, "Well, you know, I have a lot of extra work to do." That is a shortsighted view because the purpose of Capability Maturity Models and CMMI models is continuous process improvement. We feel that the CMMI model offers numerous enhancements over Software CMM Version 1.1. It should be advisable for all activities to at least have a look at it. One obvious enhancement is that you now can assess yourself in a continuous environment, as opposed to merely a staged environment. That may benefit a number of companies who don’t need to add all attributes of Level 2, but they want to do some of Level 3 or Level 4. The continuous environment allows them to better tailor the CMM to their application. Of course, with increased government emphasis on process improvement, that may provide additional incentive for adoption.
Phillips: I like Joan’s comment about the growth path and the notion that what is now in the CMMI model is in fact an excellent growth from what was in 1.1. The areas that aid the engineering discipline are now fleshed out in better ways, and I think that capitalizes on Hal’s point. Here are things that the software engineering community was doing, perhaps without adequate recognition of the quality and the ability to address these broad engineering requirements--for example, validation and verification.
Weszka: I’d like to add one other comment, and that concerns the impact to the bottom line. If an organization is in fact using the Software CMM today and another part of that organization is using another model--for example, it might be using EIA/IS 731--adopting the integrated model, we would expect, would result in improved efficiency in their process-improvement efforts. So there may be strong business reasons for an organization to move from its existing use of the Software CMM to the CMMI model, if there are other stovepiped efforts underway.
Ahern: Historically there has been an issue in a lot of places about the differences between software engineering organizations and systems engineering organizations. Even though it is essential for these two groups to cooperate--and in many instances they do, quite successfully--still they have been in different worlds. I think the CMMI model offers an opportunity for those who want to bring these organizations more closely together. It provides them with more common terminology and allows them to communicate better in an integrated environment.
Weszka: One of the biggest differences between the software engineering and systems engineering communities regarding process improvement has been the fact that because the Software CMM is a staged model and EIA/IS 731 is a continuous model, the two engineering groups have different process-improvement paradigms. By adopting one of the CMMI "architectures," either the staged or the continuous, and using the integrated software and systems engineering model, combined engineering groups can bridge some of the differences that have existed in the past and establish a more common culture for process improvement.
Rassa: A simple change like changing from the software engineering process group to the engineering process group causes a paradigm shift that may enable some of that cross-organizational building of the community of practice for broad, enterprise process improvement to be more effective.
Wilson: Most organizations do both software and systems engineering, in the sense that there are elements of systems engineering in most big software programs, even when they’re considered software only. One of the things you find when you break down the barrier and ask what’s in common is a tremendous benefit return. In the case of the integrated SE/SW CMMI model, or even just the software version, using terminology that is normally used in systems engineering raises the understanding of the software engineers to the importance of doing some of those activities. And while they’re mentioned and covered in the software product engineering part of Version 1.1, they become, I think, more defined and clear in the CMMI versions. They are effectively the same between the software and systems engineering group requirements, regardless of whether you are using the continuous or staged model in the CMMI. I think that’s its real power from a practical point of view.
Shrum: We’ve touched on systems engineering. Is there anything in particular that you would use to persuade systems engineers to transition to CMMI?
Wilson: Our organization adopted 731 as our systems engineering standard when it became available. One of the issues was that it’s a slightly different model in the sense that it’s a continuous model. What we use in Version 1.1 of the Software CMM is a staged model. So we were faced with some transition issues. From the point of view of an organization that was just going to use 731, almost every major system activity that you perform today includes some software. Because of the way we built things to meet our cost objectives--and without having a combined CMMI--it would be very difficult for a pure systems engineering view to be as effective in our business areas. There may be some organizations that indeed do just systems engineering and don’t consider software, but I think there are probably very few of them. Having used both models, I would encourage an organization to look toward a combined CMMI model as the way to move, particularly since the new version of CMMI really makes it simple to provide both because they use almost a common view.
Rassa: Raytheon also has begun to adopt 731 because it was essentially de rigeur, and we’ve done four assessments. But we find that what’s lacking is our integration of software and systems engineering processes, which is what CMMI does for us. I agree with Hal’s point that most of the government or aerospace industry really does systems engineering and software engineering, among other disciplines, and the integration activity that CMMI is expected to bring will have great efficiency benefits to us and to other companies. That was the whole notion behind CMMI to start with.
Phillips: The points that I would like to insert here build on a comment that Bob made earlier about the advantages--the various pros and cons between the staged methodology and the continuous. I’ve just dealt with an organization that has a strong systems engineering community within it, but is trying to decide how to approach the problem. They’re looking at the desirability of being able to move, if you will, vertically, which calls for the continuous notion because there are some things that they really want to be able to accelerate above others to their management, to meet their management business goals. But they also have this difficulty of making sure that they get the right ones first, which is more of the staged paradigm.
Ferguson: I agree, especially about allowing enterprise-wide improvement versus individual stovepipes in the various disciplines. The current software and systems engineering models foster improvement in each discipline, but CMMI allows both the systems engineers and the software engineers to communicate horizontally, and not just vertically. It also allows the process-improvement efforts to occur in a much more integrated fashion across the organization rather than vertically.
Rassa: Let me throw in another possibility here. It strikes me that one of the advantages of moving forward in the model is that the 731 instantiation focused, for good reasons, on those things that were considered normative and did not give more information on how one might proceed against the various practices being called out. That was a very different paradigm from what existed on the software engineering side. Now if someone says, "Give me an example, tell me a little bit more," it is available in the Volume 2 portion. So the informative Volume 2 includes the normative material and also provides the ability for folks who need more guidance on how to proceed to get it from that side, from the overall model.
Weszka: Some people would consider that a downside to the CMMI model specifically. They could say, and I have heard it said, that 731 is written in a very concise, clear way; the practices are well articulated; and there isn’t a need for a lot of explanation.
Rassa: We have the two volumes, I think, for a very good reason.
Shrum: How do you see the CMMI model fitting into the international standards environment?
Rassa: Interesting question. The CMMI Steering Group has established the general philosophy that we will look at issuing CMMI first as a national standard and then secondly as an international standard. The reason for progressing to an international standard is to give it some durability and credibility, particularly in an international marketplace, when U.S. companies are doing business abroad or partnering with companies abroad. Having CMMI as an international standard will, we believe, facilitate that process. However, I should also point out that we are just beginning the process of making CMMI an international standard, and it will be quite a number of years before that will have been consummated. A lot of arrangements will have to be made and a lot of planning with the standards organization that would do the issuance. We have initiated discussions with the GEIA concerning the national standard, the U.S. standard, and that’s as far as we have gone so far--just discussions initiated. We believe that having an international standard is advantageous; however, we have yet to assess all of the pros and cons. Before we finalize the move to an international standard, I believe we would need a lot more input and have some serious discussion about it. We have an indication that to create an international standard is a very tedious process and might disturb some of the CMMI plans that we have in place, in terms of our custodianship and how we authorize assessors and trainers.
Wilson: One of the things that the steering group and the PDT (product development team) recognized early in the process of defining the requirements for CMMI was the need to recognize the input from ISO 15504. The effort that the PDT put into mapping and adjusting the CMMI so that it would have a relatively smooth means of mapping into 15504 is important to recognize. That’s its primary means of mapping into international standards today, prior to any activity to actually make the CMMI itself an international standard.
Ahern: Indeed, we did have the requirement and we did keep our eye on that standard, and we have the goal of making CMMI consistent with it. However, it is a moving target in that the standards, not only ISO 15504 but also other relevant ISO standards, are on the move and changing as we speak. So it’s going to be an ongoing issue to keep CMMI well positioned and consistent with relevant international standards. Obviously we wish to avoid creating a situation in which U.S. companies that conduct business internationally are at a disadvantage because they work to a CMMI model. A CMMI model should not put up a barrier and should not make it more difficult for a company to be consistent with international standards. So I think it’s an important goal and one that needs to be constantly reevaluated as we proceed in the next few years.
Phillips: Let me toss in a couple of thoughts. Since I joined the SEI four years ago and began attending the annual Software Engineering Process Group meetings, I was seeing increasingly international participation, to the point that it seemed to me that the Software CMM was becoming a de facto international standard. But at the same time, things like 15504 were requiring some things that the existing model didn’t accommodate, at least not easily. So to me one of the values of our approach to CMMI has been to recognize the desirability of producing something that will meet those needs more effectively, to accommodate the 15504 intention and therefore make it a logical move from a de facto standard around this set of ideas to something that can then be embraced first as a national standard, as Bob says, and over sufficient time, potentially as an international standard.
Weszka: I agree that it is a critical goal to move the CMMI models into the standards arena. Of course, harmonization is really important, and we’ve already discussed the amount of work that has been done to date to ensure that we are harmonizing with other standards as they evolve. The SEI, as the custodian of the model, will be a key player in taking the CMMI forward into the standards arena.
Shrum: What effects will senior executives see as a result of using a CMMI model?
Phillips: Let me start with the one that I found to be so persuasive in coming to the SEI in the first place, having been an executive in the Air Force before. I was talking with a member of an engineering group who said they had adopted the CMM as an approach and had reached, in the staged model, Level 3. The effect was that the work force now understands far more clearly how the job needs to be done and there’s a stability in doing things in an understood and effective way that simply didn’t exist in the chaos that existed before adopting the models. So to me the senior executives who have not been in a process-improvement domain associated with this kind of building capabilities and maturities will see great impact on the effectiveness of the work force.
Weszka: I would agree that senior executives should observe additional business value and return on investment for process improvement as their process-improvement efforts are integrated using the CMMI models. But in addition to the tangible results, they should also see the intangible benefits accruing over an expanded segment of their business. For example, improved employee morale, reduced attrition, better communications--all those additional advantages from process improvement should be visible across a larger business base. Senior executives who have already sponsored process improvement typically would have organizational cultures that support measurement already. So they can measure the results of using the CMMI model, including improvements in quality, productivity, cycle time, and customer satisfaction. In addition, they can use channels such as employee satisfaction surveys and the number of suggestions for improvements to measure the intangible results. And, of course, they can continue to use the measurement programs that they have in place to measure the quantitative tangible improvements to their business.
Wilson: In addition, I think in an organization that has traditionally been a Software CMM supporting organization moving to a systems engineering supporting organization, one of the big-value elements that senior executives will see in the CMMI is that both software and systems engineering elements of the organization will begin to speak with a common lexicon within the process areas. Rather than saying, "Well, we’re different because...," the integrated CMM really brings together process definitions in a way that all can speak the same language. While the individual processes may vary slightly, internal to their suborganizations, the basic senior executive view will be the same, which should make it much easier to operate in that type of organization.
Rassa: I’d like to give a different view. Most senior executives don’t tend to focus on a number of the things that have been mentioned. What they tend to focus on is the bottom line, and we hope that senior executives will see an improvement in their win ratio of business pursuits due to the adoption of CMMI, which is caused by several things that some of my colleagues have mentioned. They also should note improved profit or performance because of the process-improvement aspects of CMMI that are given to each program that is won by an organization so that they can have a greater assessment of risk. And that risk should be lower because of the process-improvement methodology that we’ve put in place. But the senior executives also need to recognize that there may be a slight investment required for implementing CMMI and that this should be clearly offset dramatically by the improved win ratio and the improved profitability.
Weszka: In addition to the things that Bob mentioned, there is the improved predictability, which of course is a fundamental concept underlying all of the Capability Maturity Models. That’s another thing that executives should expect to see: improved predictability in program performance across the enterprise as process improvement is focused on a larger segment of the business.
Ahern: If you look at some of the differences between software organizations and systems organizations within a given company, in a sense the software organization is the new kid on the block. The other engineering disciplines are--and view themselves as--more well established, whereas software engineering is a new and increasingly important discipline that perhaps is still trying to discover ways to be fully accepted as a real engineering area. I think that one of the things that software engineering has to gain from adopting a CMMI approach is to become better integrated into the larger engineering context. At the same time, it seems that software engineering has taken the lead relative to other engineering disciplines; software engineers have recognized--as they have in the manufacturing area--that getting processes under control and especially under quantitative control is very valuable. Applying that, as Joan said, across the entire enterprise can potentially help a great deal in terms of improving competitive position.
Ferguson: I think that senior executives will start to see enterprise-wide process improvement expand beyond software engineering and systems engineering. The premise of CMMI is that it allows CMMI process-improvement techniques to apply to any domain. We’ve just focused on the two, software and systems engineering. I think senior executives will see that this common way of improving processes can apply to everything--not just technical development. It can apply to finance; it can apply to project management. So its use could explode dramatically from the current relatively narrow technical communities to areas across the entire organization.
Shrum: How do you see CMMI models being used by organizations, and how would you warn organizations not to use CMMI models?
Wilson: From the perspective of where you start and where you might go, obviously the concept of the Capability Maturity Models, and the CMMI models in particular, is to focus on the initial definition of the processes that make up a mature organization. Once that is established, the focus moves to improving and tuning those processes over time with the appropriate feedback. One of the things I would recommend to an organization starting out is to focus on the internal process value of the business processes that they perform and not use CMMI structure to usurp them. Because CMMI models are really intended--as I see it and as we’ve used them--as guidance documents. They help to ensure that your processes include all of the critical elements that a capable, mature organization needs, and help ensure that you don’t leave any out. The CMMI approach isn’t intended to force an organization to put some emphasis where no emphasis is demanded in your internal process. In the future, as things change and organizations adapt, you’ll find that the core processes remain. You can keep them and map them to the variations that occur. Having gone through several versions of systems engineering process models, and also going from the SW-CMM Version 1.1 orientation and moving both toward a CMMI model, we’ve found that our processes remain stable because they were good processes to start with. They were tuned over time in the Version 1.1 model and in the other systems engineering models. But we’re now mapping our internal processes to the CMMI model. It becomes a lot easier because those processes still remain our key business processes. If you look at using a CMMI model that way, you focus on your real values and the return you get from your processes. That is where the real value is achieved.
Ahern: The CMMI product development team had the requirement to create a model that would be good for process improvement across multiple disciplines, with systems engineering and software engineering as starting points. An organization needs to think about the different ways in which a model might help with process improvement. On the one hand, you could be focused on benchmarking: determining where you stand relative to either where you were two years ago or where you want to be three years hence, or vis-à-vis your competitors or perhaps across a larger industry. So one use of the model for process improvements is to try to set benchmarks for where you are and where you want to go. On the other hand, there are uses of the model that allow greater attention to detail. You might not look at the whole model but instead just use the parts of it for more narrowly focused efforts at process improvement. I think it’s important for managers to realize that they have this second option. Indeed the more narrowly focused look might have a larger payback because it allows you to take a particular area of concern, examine it more thoroughly, and do more problem identification than you would be able to do if you took the whole model, which is very broad and all-encompassing. If you only are involved in benchmarking efforts and don’t do the other, I believe you’re missing an opportunity to identify process problems and their root causes.
Rassa: In summary, I think a CMMI model is designed to be used to guide a company’s process improvement, either at an organizational level or a programmatic level. A company should recognize that at a programmatic level we do encourage tailoring to suit the specific program. You shouldn’t try to look at CMMI as the gospel, that you absolutely have to follow it to the letter. You’re encouraged to tailor it as it applies to your organizational needs. It’s supposed to guide your company’s internal processes. Most companies have internal processes, software operating instructions, or integrated product development processes, et cetera. And a CMMI model is the map to where your processes should lead. But you should still build your processes around your organizational needs.
Weszka: I agree with Bob’s points. The CMMI models should be used for internal process improvement. That includes process definition, improving the processes once they’re established, and then assessing them. I would suggest that although the CMMI models can be used strictly for benchmarking purposes, a better approach is to use them both for improvement as well as for benchmarking. On the other hand, they should not be interpreted prescriptively. That, to me, is the biggest danger--that organizations could interpret the practices as mandatory requirements. Instead, the model should be tailored to the organization’s business objectives; that must be a consideration whenever the models are used.
Phillips: I’d just like to reinforce that. We often find, here at the SEI, a desire from various organizations to have us tell them what processes to use and help them choose exactly the processes that will meet a particular process area within the existing CMMs. We seek to dissuade that because it’s so important to determine the business value, to determine the processes within the company that meet their needs and then to see how those interrelate as a cross-check against things that have been found across other industries. But we would wish to dissuade anyone from saying, "Let me just find those things that meet some particular process model and only do those particular things for improvement."
Ferguson: I’d like to go back to something that Bob Rassa said earlier and emphasize that process improvement using these models requires up-front effort and resources.Just taking one of these models and laying it on the back of the existing work force and saying, "Here, go implement this," is a surefire means for disaster. Resources have to be provided to define the processes, define the process improvement steps, and implement those. We find that over and over again in the government that adequate resources are not provided up front. We look at the anticipated savings, but don’t provide the seed money to gain those anticipated savings.
Rassa: CMMI models should not be used solely to achieve a rating to satisfy a perceived customer requirement.
Phillips: I think we’d all agree with that. That’s just a clear statement of the potential misuse. The value of the CMMI effort is to improve the processes within the organization. Benchmarking is desirable, but it should not drive the result.
Wilson: I think if you look at the long-term value of following a Capability Maturity Model or a CMMI model, the organization changes with the beginnings of process improvement. As Jack pointed out, if you initially invest correctly and focus on the value received, the improvement you make, and how that improvement actually adds value to your organization, you will make the right start. Then the question of measurement of internal progress, adapted based on which model you choose and which representation you choose, does have value internally because you’re always comparing against a common element: your internal view. But that’s not the real reason you do it. You measure because you watch the results and see the value returned, not the number assigned to a level. The number or level is really an indicator of how well you’re progressing internally. Are you achieving the milestones and time stages you’ve set in terms of your overall improvement plans? In that sense it’s a valuable tool. I believe that misuse occurs when you start to compare unlike elements. Even within a corporation, where multiple organizational entities are independently assessed, it’s very hard to compare results from one entity to another. They may be using a common organizational process, but tailoring processes to specific organizations makes comparison very difficult, even within a single corporation.
For more information
Please tell us what you
think with this short
(< 5 minute) survey.