NEWS AT SEI
This library item is related to the following area(s) of work:Process Improvement
This article was originally published in News at SEI on: March 1, 2000
My December 1999 column described how to make the strategic case for process improvement. In this column I provide an example of how to do this. This column thus assumes that you have the ear of a manager or executive who thinks strategically and will consider investments that will likely take a few years to pay off. In the next column, I will talk about dealing with tactically focused managers.
The financial justification process has five phases:
My December 1999 column generally discussed these steps. This column walks you through a hypothetical case study where Tom Jones develops a proposal to introduce the Team Software Process (TSP).
Tom reviewed the situation in his organization and found that management’s top priority was to reduce development cycle time. He decided to do this by introducing the TSP. He also talked to experts about the TSP introduction strategy and found that it had the following 7 steps:
Hold an executive seminar for selected top managers and executives from the division or laboratory. Tom estimated that there would be 20 attendees.
Give a one-half-day planning session to determine the improvement plan. Tom assumed that 10 of the first-day attendees would participate
Tom assumed that these first two phases would be successful but decided to include a decision step to reduce the required initial commitment.
Train the involved managers in the Personal Software Process (PSP) and TSP management methods. Tom planned to include the team leaders, the managers of the team leaders, and several other managers below the executive level. He assumed there would be 10 managers in this course.
Provide PSP training to all the engineers who will be on the teams. Tom assumed there would be two teams with eight engineers per team, or 16 engineers.
Provide general PSP training to any team members who are not programmers. This would be systems, hardware, or test personnel, for example. He assumed that the first two TSP teams would be software only and that there would be four system requirements team members in this category.
Tom planned to train two engineers to be PSP instructors so that they could support the two TSP teams, handle training of any team turnover replacements, and support further TSP introduction. These two instructor candidates would also attend PSP training in step 4, bringing that total up to 18.
The final introduction step Tom planned was to launch the two TSP teams.
Assuming that these initial team launches were successful, Tom planned to ask management to proceed with broader TSP introduction.
Once Tom defined the proposed introduction program, he estimated its costs in four parts:
For the labor costs, Tom estimated the number of people to be involved in each introduction step as shown in Table 1 and Table 2. Because the TSP launches in step 7 would be part of the project, however, he did not count them as training time.
|Step||Item||People||Prep. Time||Class Days||Engineer Days||Manager Days|
|3||Manager PSP Training||10||0.3||4.0||43|
|4||PSP Course I & II||18||2.5||7.0+7.0||297|
|5||General PSP Course||4||0.5||3.0||14|
|6||Instructor/ Coach Courses||2||10.0||20|
Next, Tom checked with the financial people and found that the cost for a day of engineering time was about $1,000 and that a manager or executive day cost about $2,000. While these rates seemed high to him, finance explained that they included all the costs for overhead, support, vacation, medical benefits, sick-time, workmen’s compensation, insurance, retirement, Social Security, and taxes. Using these numbers, Tom calculated that the labor costs to train the two TSP teams and their managers would be $331,000 + $136,000 = $467,000.
|Cost Item||Cost calculation||Total cost|
|Internal introduction costs||
$1,000*331 engineering days$2,000*68 manager days
|TSP coaching costs||2 engineers*40 days*1 year||$80,000|
|External introduction costs||25% to 75% of internal costs||$136,750 to $410,250|
|Total 1-year costs||$683,750 to $957,250|
|TSP coaching costs||2 engineers*40 days*4 years||$320,000|
|Turnover training||3 engineers*14 days*4 years||$168,000|
|Total 5-year costs||$1,171,750 to $1,445,250|
Tom found that each TSP team would need coaching support of about 20% of the time of a TSP instructor/coach. These costs would add about 80 engineer days per year or $80,000 a year.
Finally, any new or replacement engineers on the teams would have to be trained. Assuming an annual turnover of 20 %, that would be about three engineers to train per year at 14 days of training each, or another 42 days of training and $42,000 per year.
The costs for the external instructors and consultants for any improvement program are typically fairly large, but they are generally much smaller than the labor costs. These external costs would include delivering the courses listed in Table 1, launching and supporting the initial TSP teams, several team relaunches, and occasional consultation and assistance.
Without going out for external quotes, Tom assumed that the external support costs for this initial effort would add between 25% to 75% to the first-year labor costs, with the level of introduction cost determined by how rapidly management wanted to introduce the TSP.
Tom realized that while the engineers and managers were being trained they would not be developing or supporting products. To account for these costs, he would reduce the anticipated first-year cycle-time improvement by the three-week engineer training time.
In estimating the improvement benefits, Tom found data on the benefits other organizations had obtained with the TSP, and he also obtained data on the current performance of his organization. Finally, he used these data to estimate the likely improvement benefits for his organization.
Tom learned that Hill AFB, on its first TSP project, increased productivity by 123% over the same team’s prior project [Webb]. Hill AFB also cut test time from 22% to 2.7% of development time, or by 88%.
He also found a presentation that Teradyne had made on its TSP results [Musson]. Through the use of the TSP, Teradyne cut final test and field defects from a rate of 20 defects/KLOC to 1 defect/KLOC. Historically, final test and field defects had cost Teradyne an average of 12 engineering hours each to find and fix.
By using the TSP, Tom also learned that Teradyne had reduced engineering and customer acceptance test time from nine months for an earlier project to five weeks for the TSP. Engineering and acceptance test defects were cut from 5 defects/KLOC to 0.02 defects/KLOC, with no further defects found by the customers.
In addition, he found that a Boeing TSP project had cut test defects by 75% from the prior project average and reduced system test time by 96% [Vu].
To calculate the TSP benefits for his organization, Tom next needed data on how much time these development groups spent in test, defect levels in the various test phases, and the cost of diagnosing and fixing each defect. With these data, he could then estimate the likely savings from introducing the TSP.
Tom assumed that the 16 engineers on the two trial projects would develop a total of about 80,000 LOC in 12 months. He also found that the time typically spent in integration and system test was currently about 40% of the development schedule and that the defect levels in test and field use were much like those at Teradyne, or 20 defects/KLOC. He assumed that these 20 defects/KLOC were split with 10 in integration test, 5 in system test, and 5 after product shipment. He also assumed that the engineering cost to diagnose and fix these defects was about 1.5 days each.
As shown in Table 3, Tom now estimated the likely savings. First, for the reduction in test defects, he projected that the integration and system test defect/KLOC would be reduced from 15 to 1, for a savings of 14 defects/KLOC. For the total 80 KLOC planned for these two projects, he calculated that this would save 14*80 = 1,120 test defects. At a cost of 1.5 engineering days for each defect, this would be a reduction of 1,680 engineering days or $1,680,000.
|1||Integration/ system test defects||- 1,120 defects||1,120*1.5 =1,680||$1,680,000|
|2||Test time||- 4.1 months||4.1*21*16 =1,378||$1,378,000|
|3||First-year savings (#1 or #2)||$1,378,000|
|4||First-year costs (Table 2)||$957,250|
|5||First-year ROI (100*#4/#5)||144%|
|6||5-year development (#3*5)||$6,890,000|
|7||Project maintenance savings||-398 defects||398*1.5 =597||$597,000|
|8||5-year maintenance (2.5*#7)||$1,492,500|
|9||5-year savings (#6+#8)||$8,382,500|
|10||5-year costs (Table 2)||$1,445,250|
|11||5-year ROI (100*#9/#10||580%|
To check these savings, Tom also looked at test time reductions. The Boeing results showed a test-time reduction of 96% and Hill AFB reported an 88% reduction. Tom assumed that the TSP would reduce his organization’s integration and system test time by 85%, or from 4.8 months to 0.7 months for a 12-month project. This 4.1-month savings, at 21 working days per month and 16 engineers, came to a savings of 1,378 engineering days. At the typical engineering day rate of $1,000, the test time savings were then $1,378,000. Because this estimate was a little lower than the $1,680,000 savings based on defect reduction, he decided to use the lower number in the justification calculations. He then compared the $1,378,000 one-year savings to the maximum one-time improvement cost of $957,250 to give a one-year return on investment of 144%.
In addition, Tom also felt that there would be a maintenance cost reduction. For field and customer defects, the Teradyne data showed a reduction from 5 defects/KLOC to 0.02 defects/KLOC. For the 80 KLOC planned in his organization, he estimated a reduction of 398 customer-reported defects. At 1.5 engineering days each, this would be a maintenance-cost savings of $597,000 for each year of development work. Because the maintenance savings would not accrue until after product shipment and would be spread over several years, Tom assumed that one fourth of these savings would accrue every year for the four years after each product was developed.
Finally, Tom assumed that the initial two TSP teams would continue to use the TSP on subsequent projects. For the five-year period, the development savings would be $6,890,000. In addition, there would be maintenance savings, but because these savings would be spread over four years, the first project would accrue $597,000 savings, the second year project would only save 3/4ths of that, and so forth. Thus, over five years, the maintenance savings would be 2.5 times the single-project maintenance savings, or $1,492,500. The total five-year savings would be $8,382,500. Because, as shown in Table 2, the total five-year improvement cost would be $1,445,250, this gives a return on investment of 580%.
Next, Tom used the reduction in test time to estimate cycle time improvement. With the assumed 12-month project schedule and 40% of the schedule spent in test, normal testing time would be 4.8 months. By assuming an 85% reduction in test time, he estimated that the 4.8 months would be reduced to 0.7 months. Thus, the typical 12-month schedule would be cut to 8 months, or about a 32% cycle-time reduction. For the first year, he also reduced this 4.1-month cycle time improvement by the three-week initial team training time, for a net of 3.35 months schedule savings.
At this point, Tom had completed the estimating work and needed to produce a brief management presentation. He decided on the following seven-part outline:
Opening summary. On one chart, he briefly summarized the problem, how he proposed to address it, and the likely benefits.
The problem: To improve cycle time and not increase costs
The solution: Introduce the Team Software Process (TSP)
The proposal. He briefly described the proposal.
He first asked for a minimum commitment to steps 1 and 2 of the introduction program.
Second, assuming that the first two steps were successful, he proposed to proceed with the 2-team TSP pilot program.
Third, after the two teams were underway and the early experience was satisfactory, he planned to proceed with broader TSP introduction.
Description of the TSP. As backup, he prepared a brief description of the TSP and its objectives.
Summary of the TSP benefits obtained by other organizations. Also as backup, he prepared a summary of the available data.
Summary of the introduction plan. As backup, he made a list of the principal phases and decision points in the introduction plan.
Summary of the estimated cost savings. As backup, he made a summary of the cost savings calculations with the key assumptions and supporting data. He also reviewed these figures with finance before the presentation.
Summary of the estimated cycle-time reduction. As backup, he included a summary of the cycle-time improvement calculations with the key assumptions and supporting data.
Tom’s final step was to make the presentation and get the order.
While this example is for a specific process improvement method, the principles are quite general. To relate this example to the discussion in the December column, the five phases in this example relate to the topics in the December column as follows:
Hopefully, this example will help you make your own business case. If you have questions, I suggest you look at the December 1999 column, which contains a more generic discussion of these same topics.
In the next issue, we will discuss the issues of convincing tactically focused managers and executives to start a process improvement program. Following that, a subsequent column will deal with how to move from a tactically to a strategically based improvement program.
In writing papers and columns, I make a practice of asking associates to review early drafts. For this column, I particularly appreciate the helpful comments and suggestions of Sholom Cohen, Frank Gmeldi, Julia Mulaney, Jim Over, Mark Paulk, and Bill Peterson.
In these columns, I discuss software issues and the impact of quality and process on engineers and their organizations. I am, however, most interested in addressing issues you feel are important. So please drop me a note with your comments, questions, or suggestions. I will read your notes and consider them when I plan future columns.
Thanks for your attention and please stay tuned in.
Watts S. Humphrey
[Musson] Robert Musson, presentation at the 1999 Software Engineering Institute Symposium, Pittsburgh, PA, August 30 to September 2, 1999.
[Vu] John Vu, presentation to the U.S. Department of Defense on the Boeing process-improvement program.
[Webb] Dave Webb and W. S. Humphrey, "Using the TSP on the TaskView Project," Crossstalk, vol. 12, no. 2, February, 1999, pp. 3 - 10.
Watts S. Humphrey founded the Software Process Program at the SEI. He is a fellow of the institute and is a research scientist on its staff. From 1959 to 1986, he was associated with IBM Corporation, where he was director of programming quality and process. His publications include many technical papers and six books. His most recent books are: Managing the Software Process (1989), A Discipline for Software Engineering (1995), Managing Technical People (1996), and Introduction to the Personal Software Process (1997). He holds five U.S. patents. He is a member of the Association for Computing Machinery, a fellow of the Institute for Electrical and Electronics Engineers, and a past member of the Malcolm Baldrige National Quality Award Board of Examiners. He holds a BS in physics from the University of Chicago, an MS in physics from the Illinois Institute of Technology, and an MBA from the University of Chicago.
The views expressed in this article are the author's only and do not represent directly or imply any official position or view of the Software Engineering Institute or Carnegie Mellon University. This article is intended to stimulate further discussion about this topic.
For more information
Please tell us what you
think with this short
(< 5 minute) survey.