columns watts new

This is the sixth column in a series on large-scale development work. The prior columns covered several important preparatory topics, but with this column the focus switches to process issues and the methods for actually doing the work. Part I introduced the structural problems of the massive organizations that typically do large-scale work. Part II addressed the management decision process, why this process is critical for large-scale work, and how to fix it if it is broken. Part III discussed how to get things done in a bureaucratic organization, and Part IV addressed the need for teams to take charge of their own work. Part V then described self-directed teams and how their planning and self-management skills help them to consistently produce superior project results.

The final columns in this series deal with actually doing the work. Once you have a suitable organization structure, management style, and work environment, two challenges remain: how to select the right process and how to get people to follow that process in doing the job. This column addresses the process question.

The Process Problem

The current situation with large-scale system-development processes can be summarized as follows:

• Large and complex computer-based systems are now critical to the economic and military welfare of the United States and to much of the developed world.
• With current methods, development of these systems is typically late and over cost and results in poor-quality products.
• Since future development programs will be far more challenging, the methods of the past will produce even worse results and often will not deliver any products at all.
• To address this problem properly, organizations must adopt sound processes that can be scaled up to the larger development challenges of the future.

Selecting a Process

Large and complex computer-based systems are now becoming widely used in the United States and in much of the developed world. If history is any guide, developing these systems with the methods of the past will almost certainly yield unsatisfactory results. These projects have almost always failed because of project-management problems. While the solutions to these problems are known and have proven to be highly effective, they are not yet widely practiced. Therefore, to successfully develop the even larger and more challenging systems of the future, we must start to use processes that address these historical problems. To do this, the processes must meet five requirements.

1. control development cost and schedule predictably
2. handle changing needs responsively
3. minimize the development schedule
4. be scalable
5. produce quality products predictably

Cost and Schedule

To control cost and schedule predictably, organizations must use processes that do five things:

1. Have the people who will do the work estimate and plan that work.
2. Track the progress of the work precisely and regularly.
3. When progress falls behind plan, promptly identify and resolve the causes.
4. When the requirements change, re-estimate and revise the entire plan promptly.
5. Anticipate and manage risks.

Changing Needs

To handle changing needs responsively, it is essential to do the following:

1. Examine every proposed change to understand its implications for the development plan.
2. Pay particular attention to each change's impact on completed work, including requirements, design, implementation, verification, and test.
3. Estimate all of the cost and schedule consequences of making the needed changes.
4. If the cost and schedule implications are significant or if they exceed the currently approved plan, get management approval before proceeding.

Minimize the Development Schedule

The proven methods to minimize the development schedule are straightforward but not always easy: increase the project staff, minimize the amount of rework, and reduce the amount of work. Rarely are there questions about how to increase project staff or reduce the amount of work; the problem is in actually implementing the obvious solutions. Minimizing rework, however, is a quality problem that is more challenging to solve. Here again, there are known and proven methods, and they rest on five principles:

1. It costs more to build and fix a defective product than it would have cost to build it properly the first time.
2. It costs more to fix a defective product after it has been delivered to users than it would have cost to fix it before delivery.
3. It costs more to fix a product in the later testing stages than in the earlier design and development stages.
4. It costs less to fix product requirements and specification errors in the earlier requirements and specification stages than in the later design and implementation stages.
5. It is least expensive to prevent defects entirely.

A Scalable Process

The fourth requirement for a process to be suitable for large-scale system-development work is that it be scalable. To be scalable, a process must meet the following three criteria:

1. It must use robust and precise methods at all levels, especially at the working systems-engineer and software-development level.
2. Technical and project decisions must be based on and give greatest weight to the knowledge and judgment of the development-level professionals.
3. The process must consistently use data that are derived from accurate, precise, and auditable process and product measurements.

These three points are not generally as widely understood as the cost and schedule problems and call for further elaboration.

Method Scalability--A truly fundamental problem in developing the large-scale systems of the future is that commonly used development methods are nearing their scalability limits and will likely be incapable of handling the much larger challenges of the future. As system scale increases, new methods are generally needed, but the smaller scale methods used in building the supporting system foundations must also be suitable. For software, the principal concern is with the design and quality-management methods commonly used. When developers design and implement the modules and components of massive systems, they typically use the same methods they used with small stand-alone programs. These methods are often little different from the methods they used to write the practice programs they developed when first learning to program. That would be like using the same practices to build the foundation for a 100-story building that you used in building a one-story structure. However, in software development, we do not just scale up by 100 times, we often scale up by 1,000 or 10,000 times, and we typically continue to use the identical development methods, regardless of the job's size.

Management Scalability--Management scalability concerns the estimating, planning, job assignment, tracking, reporting, and control of development work. The current common practice is for managers to make the plans, allocate the work, report progress, and provide project control. However, the suitability of project planning, work allocation, tracking, and control decisions depends on the level of available project information. Furthermore, the level of available project information is inversely proportional to management level. It is therefore clear that this common management-centered planning, allocation, tracking, and control process will not likely work very well when scaled up to support massive systems-development programs.

Measurement Scalability--Why is it that financial accounting methods work for very small organizations and are equally effective for very large corporations? The fundamental reason is that the financial community uses precisely defined methods and auditable data. These methods also assume that the data can be in error, and they include consistent auditing and checking practices to ensure that the data at every organizational level are both accurate and precise. Few of the methods commonly practiced in the development of software-intensive systems use data of any kind, and those that do typically rely on data gathered by other groups, such as accounting, testing, and field support. To have a scalable process, all of the management, technical, and quality practices used in that process must use data that are derived from complete, accurate, precise, and auditable process and product measurements.

Predictable Quality

The fifth and final requirement for a process to be suitable for large-scale work is that it be able to produce quality products predictably. This is one of the greatest challenges faced by the software community today. The problem is that current practices do not acknowledge that no testing program can identify more than a small percentage of the defects in a large and complex product, and that the larger and more complex the product, the smaller this percentage will be. Further, this means that to get a high-quality product out of testing, one must put a high-quality product into testing. The requirements for a process that will do this are as follows:

• The process must include a family of early defect-prevention and defect-detection activities.
• The process must define and use measures that verify process quality during process enactment, and these measures must be applied at every management and development level. This will ensure that all or almost all product defects are found and fixed before testing begins.
• The process must also include measures that verify product quality, both before and after testing.
• The process must ensure that quality plans are produced and reviewed regularly, and that deviations from these plans are detected and corrected promptly during process enactment.

Today's commonly used software-development processes do not incorporate quality measurement and analysis. This is a crucial failing since, with even rudimentary measures, the solution to the software-quality problem would have been obvious, and sounder and more efficient software-quality practices would have long since been adopted. The simple fact is that without measurements, no serious quality program can be effective. While developers can make rudimentary quality improvements without measures, to achieve the defect levels required in modern complex systems, we must strive for defect levels of a few parts per million. Such levels are not achievable without complete, consistent, precise, and statistically based quality measurement and analysis.

Conclusions

Development processes must meet the following requirements to successfully produce the large, complex, and critical systems of the future:

• control development cost and schedule predictably
• handle changing needs responsively
• minimize the development schedule
• be scalable
• produce quality products predictably

Finally, the most critical point of all is that a process is of no value if people do not use it. This is the topic for the remainder of this series on large-scale work. For further information about how to improve large-scale systems-development processes, see my SEI technical report on the subject, Systems of Systems: Scaling Up the Development Process [Humphrey 06].

Acknowledgments

In writing papers and columns, I make a practice of asking associates to review early drafts. For this column, I particularly appreciate the helpful comments and suggestions of Dan Burton, Bob Cannon, David Carrington, Marsha Pomeroy Huff, and Bill Nichols.

In Closing, an Invitation to Readers

In this series of columns, I discuss some of the development issues related to large-scale projects. Since this is a complex subject, I cannot hope to be comprehensive, but I do want to address the issues that interest you most. So, if there are aspects of this subject that you think are particularly important and would like to see covered, please drop me a note with your comments, questions, or suggestions. Better yet, include a brief anecdote that illustrates your ideas. I will read your notes and consider them when planning future columns.

Thanks for your attention and please stay tuned in.

Watts S. Humphrey
watts@sei.cmu.edu

Reference

[Humphrey 06]
Humphrey, Watts S. Systems of Systems: Scaling Up the Development Process (CMU/SEI-2006-TR-017). Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University, 2006.

About the Author

Watts S. Humphrey founded the Software Process Program at the SEI. He is a fellow of the institute and is a research scientist on its staff. From 1959 to 1986, he was associated with IBM Corporation, where he was director of programming quality and process. His publications include many technical papers and several books. His most recent books are Introduction to the Team Software Process (2000) and Winning With Software: An Executive Strategy (2002). He holds five U.S. patents. He is a member of the Association for Computing Machinery, a fellow of the Institute for Electrical and Electronics Engineers, and a past member of the Malcolm Baldrige National Quality Award Board of Examiners. He holds a BS in physics from the University of Chicago, an MS in physics from the Illinois Institute of Technology, and an MBA from the University of Chicago.

The views expressed in this article are the author's only and do not represent directly or imply any official position or view of the Software Engineering Institute or Carnegie Mellon University. This article is intended to stimulate further discussion about this topic.

Return to COLUMNS list