discussion about your experiences with training engineers to use the Personal Software

engineers and—for those of you in academia—students receive these new ideas. To start

with, give us a brief description of your experience with PSP and TSP.

Mellon as guinea pigs for the first PSP course back in 1994. Their data appears in his

then and we require it prior to students arriving on campus. After they’re admitted, I send

They take the course before they get here because we use TSP on all of our project teams

in the software development studio. The studio is 40% of the curriculum for the year. We

had one team last year and four teams this year.

was on the Embry-Riddle advisory board and he convinced us to try out the draft that he

was working on. One of my colleagues taught it. We thought it worked pretty well and

it’s now part of the core—the first courses that entering master of software engineering

students take. We’ve been using it for about four years.

A couple of years later, Watts was convinced that maybe we needed to introduce PSP at

sometimes called “PSP Lite,” in our freshman sequence of courses. We’ve been using

that for about three years with some modest success.

Last year, we started using TSP in the junior/senior-level course Introduction to Software

Engineering, which is a broad-brush course with a project. For the first time this

semester, we used it in our undergraduate senior design course.

I also worked at the SEI as a visiting scientist a couple of years ago in the process

program and worked with Jim Over and Dan Burton, helping to develop some of the

instructional material for the first PSP courses. I sat through the instructor’s course. For

the last couple of years at Embry-Riddle, with the help of the SEI, we’ve developed and

delivered a one-week summer workshop for faculty on PSP. Some of my colleagues have

worked with Motorola and Boeing and several other organizations, helping them

introduce PSP into their workplaces. I’ve had some involvement with that.

when we were looking for answers on how to apply the Capability Maturity Model® for

Software (SW-CMM®) to small software projects or small software teams. We had been

struggling with CMM. Watts Humphrey recommended that we take a look at PSP as a

way of trying to progress and solve some of the issues. I was involved with the initial SEI

train-the-trainer course in fall 1994. It’s a required training course for our software

engineers and project managers at our organization.

Currently, we have two sides to our business, the development group and the consulting

group. As of yesterday, we have 100% of the people in our development group trained.

Training in our consulting group is a little bit more spotty. We’re trying to solve the

issues in terms of how to get people to use PSP effectively when they’re the only people

who have been trained in it and when they’re off working at a customer site and the

customer has very chaotic processes, if anything at all.

At this stage, we have not utilized TSP, primarily because it was not around in 1994

when we were dealing with the issue of how to integrate PSP into our software

development work. Somewhere down the road, we will give it a shot. It’s a timing

issue—we’re trying to figure out which project to introduce it with.

South Carolina Research Authority. I was trained at an industrial site, which is fairly rare

because most PSP instructors have actually been trained in Pittsburgh. There was a

special arrangement that allowed some instructors to be trained off site. Since then, I have

taught the PSP class at my university. I intend to do that again in the fall and follow that

with a Team Software Process course in the spring.

I had a tremendous opportunity this year to work with the PSP/TSP group. I’ve been

involved in teaching virtually every one of the PSP and TSP offerings at the SEI. The

opportunity to work at an industrial site has been extremely valuable for me. I’ll be the

coordinator for a summer faculty workshop, where we’ll be doing essentially the same

course that Tom did for the last two summers to train faculty to teach PSP. We’re also

going to have a separate workshop to help faculty plan to introduce TSP into their

curricula, or at least to give a course based upon the new Team Software Process book by

Watts Humphrey.

SEI’s Software Engineering Measurement and Analysis (SEMA) Group to help collect

data initially from the training courses and now from the field where engineers are using

PSP on the job. I’m helping to characterize the impact, using some fairly rigorous

statistical techniques, of what happens when you use the technology.

I’ve had occasion to visit with and interview a lot of folks in industry. I’ve also gone

through the training myself and am currently applying some of the methods to my work.

Though I don’t develop software, a lot of the discipline methods and data-collection

techniques are actually quite applicable to a variety of settings. I’m using it in the impact

study work that I’m doing now.

interested in PSP because I taught a little bit of it in my classes at the graduate level for

master of science students. I didn’t get into the full-blown course, but I did show it to the

students and it was quite interesting to them. Then when I started working with Gina

Green on her doctoral dissertation, we selected PSP as an innovative technology to study

our research model.

study in my doctoral dissertation, which looked at software development technique

diffusion.

For those of you working with students, how would you say they are being received by

students? Also, could you describe some of the circumstances that contribute to success

with those groups.

how we’re actually teaching the process—we’re not really teaching the programming

style—people are a little bit more oriented toward picking up the points. There is a lot of

process for relatively small problems during the training. Sometimes the students focus

on that instead of seeing the end result. We’ve had some problems giving appropriate tool

support for PSP. Our spreadsheets haven’t been that good, but that’s probably our fault.

On the other hand, using the TSP spreadsheet program that Jim Over developed—and

which is still being developed now—has worked very well for us. The detail I get from

that has been exceptional. Since we have a collaborative tool that we use on the Web to

keep track of all of our data and products within the various projects, the mentors at the

SEI and myself can see what’s going on at any given time. We put those spreadsheets out

there so that we can see each cycle—we can see how things are going along. We can see

earned value, actual and predicted, the amount of time they’re spending, how good their

estimates are. I can, on an individual basis, tell how somebody may have messed up in

underestimating or overestimating. Then we can talk about that because we have regular

meetings with individual students. So, we’ve found in cases where there are tools

available and where people understand that they’re learning a process, it seems to be

pretty successful. If you leave them to their own, to invent their own tools or to spend too

much time concentrating on the problems themselves, that doesn’t seem to work.

very well. I’d say 90% of them think it’s great and they’re able to complete the PSP

material. The difficulty comes on the job: they’re going into embedded systems, trying to

apply PSP or use it for maintenance. Adapting it to their job has been difficult. Some of

our faculty have worked to help them, mentor them, support them, and coach them.

That’s helped, but that’s the difficulty, actually bringing it into the workplace. This is a

place where management supports PSP. There were similar problems at Boeing. In the

classroom setting PSP, at the graduate level, goes over pretty well. Then we ask students

to use it in some other courses. It hasn’t always been that effective. In the freshman-level

course, we struggle with how to motivate students and produce pretty good results. The

problem in our freshman courses is that students can’t really see the improvements that

you get in a typical PSP course. The training effect has been pretty good, but the followup

in using it has been a little bit of a problem.

We started using TSP about two years ago and did two graduate student projects. Watts

came down and helped us launch development of a new scheduling program for our flight

line. We did a database program for the county assessor’s office. They were both very

successful. They used early versions of the TSP and I think students really started to see

the value of PSP. I’d say those were successful and students were able to follow the

process and use the process to their benefit.

Software engineering is a junior/senior-level survey course. It usually has some sort of

team project involvement. I’ve been doing this for about eight years with marginal

success. I used TSP last fall and it was the most successful as far as students learning to

work as part of a team to develop a piece of software. In this case, it was a toy project,

still didn’t like filling out all the forms and collecting all the data, but they grudgingly

admitted that it was worthwhile to do that. I did hear comments from a number of

students, “Now I see why we study PSP.” It has been, I think, a grand success so far.

Not too many things are questioned anymore in terms of what we’re trying to do with

PSP and the engineers’ acceptance of it. Everyone in the organization is doing it on the

projects that we have management control over. Obviously we’ve got strong management

ownership and support to make sure that it gets done. We’ve done quite a few things to

integrate PSP into our culture to where, I think, the engineers really appreciate it, even

though they still don’t like filling out some of the forms. They really do appreciate how it

helps them during the development work.

My experience has been in working with our own software development staff and with

outside organizations. It’s not uncommon at all—especially for experienced engineers—

to be resistant, early on, to all the planning and all the forms you’ve got to fill out. But

90% of the time the data that’s being collected and the feedback they get from that tends

to turn the disbelievers into believers by the time the course is over. We recognize that

there’s a certain class of people who will never buy into these things no matter how much

you try to convince them.

The theme of filling out forms seems to be a common, recurring issue. During PSP class,

I have to spend time reintroducing most engineers to what a pen and pencil are. I had one

engineer claim that he would get hand cramps from filling out the forms. Even though the

course gets better and the support tools get better, there’s still room for us to work on

things and to make them even more acceptable. To a certain degree, a tool will ease that

transition for people.

Other interesting reactions I get from students—especially as I start working with

organizations that are much larger than our own—is that although many of them

understand the value of PSP, they have total disbelief that their management will be able

to successfully pull it off. They think that everything will go fine until the first schedule

crunch comes around, and then managers will tell them to push all of this stuff aside and

get back to it some day when they have more time on their hands. Organizations of that

size are not far enough into implementation yet to find out whether or not that’s actually

going to be the case.

reasonable tools and other people developed their own tools. We also got a number of

comments like, “We can’t be very productive with this unless the managers and

development team agree on a set of tools that we’ll all use.” So, there is still some flux in

the field about what are the right tools and who should supply those tools.

that 10% of the people who just don’t want to do it. They’re searching for excuses. On

the other hand, I think we can appreciate that handy tools make it a lot easier for the

amount of data that PSP wants us to collect.

about myself and about the way I tend to work.” There were others who would never

really want to use PSP unless they were in an environment where it was expected of

them. I taught an industrial course, where unfortunately there was little management

support, for programmers who didn’t work in the traditional programming environment.

They were not all familiar with even C or Java or Pascal, for example. They kept insisting

that because the problems were irrelevant to their work that they couldn’t see why they

should be doing it. The effort to motivate and say “We should all be working on common

problems to collect data” was not one that was effective with them. If I ever worked in

such an environment again, I’d want to be very careful to determine with management

that they’re much more supportive. In teaching an industrial course here at the SEI, I

found a very positive reaction to PSP by an extremely skilled group of programmers.

seemed logical that the software industry would want to take this up—why wouldn’t they

want predictability? But it’s very difficult to really be honest with yourself and to instill

that discipline to change the way you work. It means recognizing that you’ve got to keep

your attention on it—you can’t take a holiday from it. But to be that attentive to what

you’re doing requires a tremendous amount of concentration and discipline. I found it to

be a very big challenge to use these methods. When I’m interviewing people in the field,

I now have a different appreciation for what they’re doing. I really enjoyed the course,

I’ve seen people succeed very well with paper and pencil—they don’t need a tool. I’ve

also seen people with the greatest tools who don’t collect the data because they don’t

want to be honest with themselves. I’ve seen people who were the only PSP-trained

people in a chaotic group project and still hold on. I’ve seen groups of people working

together who really benefit from having each other to talk to. At the same time, within

organizations where there are groups that are successful, you’ll find other engineers who

struggle initially to try to keep the data, try to keep their investment in what they’re

doing. But when a management fire drill comes along, the message comes to them very

clearly that spending time collecting data and getting better as a professional is not a high

priority. Making this delivery is a higher priority. Senior management and middle-level

managers can make that message very clear to engineers without having to be very

explicit about it. That’s one of the things that really gets in the way. I think the comments

we’ve heard thus far really support that as well.

for computations and rolling up the data. That’s not the hard part. The hard part is that

discipline—that relentless, continuous observation of the way you go about your work.

Recording the time and defects just requires attention to details.

reluctance to. They claim that they’ve got to have some sort of stopwatch tool in the top

right-hand corner of their window or monitor or they just can’t do it. It is perceived as too

much overhead to write down on a sheet of paper something like, “It’s 1:40 and I’m

starting to code.”

surveyed software engineers about their experiences with PSP. Could you summarize that

project and your findings?

when new technology gets put into place in organizations. Most people have looked at

end-users of technology. We wanted to look specifically at software development groups

and individuals and analyze a good innovative idea that they were using. We came on the

idea of using PSP because of its newness to the market and the idea that it was an

individual type of technology as opposed to a group or a team technology. Most of the

past research was on individuals, so this would allow us to relate what we found to some

past research in this area. The model that we developed is in the SEI special report.

sense of personal control in the software development process is very important. In fact,

we found that that was strongly related to the developer’s overall satisfaction with using

PSP and the developer’s use of PSP. PSP tended to provide a pretty high sense of

personal control to the software developer. That was good news from a theoretical

standpoint. From a practical standpoint, we chose PSP because there was a lot of interest

in it, but we found that the actual use of PSP was not as widespread as we had

anticipated.

project for at least three months and not more than two years. That made the use of PSP

an innovation. With that constraint, we eventually got good surveys from about 72 users.

software that they developed. In addition, they felt that PSP made them more conscious

of quality-related issues in software development. This means that PSP is fulfilling some

of its intended goals, at least as far as the respondents’ perceptions go.

using PSP for projects together. What do you think is the importance of keeping team

members together both in the training and then in the project work?

“pressure” is the right word—maybe peer acceptance might be a better word. When we

introduce PSP, there are some who want to do it right away and some who don’t want to

do it. There are some who sort of will do it if everyone else is doing it. Of course, we

have a clear advantage in that I can just order everybody to take the course. I could even

pay attention to whether they’re using it later. But I still think that they’ve been much

more effective when they could use it as a group, when they could talk to each other in

the same language.

it was difficult to implement because their work was team-based. Just looking

specifically at PSP, developers found that there was a disconnect between individual- and

team-based training.

beginning mechanics of building an effective team. In industry or academia, these team

projects often don’t have a process. They may have an organizational process, but not a

process that’s attuned to teamwork. TSP guides you through some elementary things that

you’ll want to do in the beginning, launch, planning, and strategy phases as a team.

Students can look ahead through a whole project and have some idea of what they’re

going to be doing week by week. This provides some assurance and a basis for the team

to start understanding what they’re going to do and communicating about that. Like so

much of PSP, it’s really not that astonishing as far as the complexity. A lot of it is just

common sense and good guidance in a structured form. I think that’s as important as

anything else in building a good team.

Unless everyone on the team is trained—from the project manager through all of the

engineers, and especially the project manager, being the leader of the team—it doesn’t

get used. That’s generally a difficult and tricky thing to do because often organizations

are reluctant to shut down a project for a month or so as they get a team trained. They’d

much rather steal one person from each of their teams and have them go through it. Then

a few months later steal another person from each of their teams and have them go

through it. Then, maybe, a year or so down the road, they can have everyone trained.

That, in itself, leads to all kinds of additional problems. It’s hard to pilot things like that

unless everyone’s ready to go at the same time. Our experience is that biting the bullet

and doing it in teams is the most effective way.

great deal of time figuring out what to do. To have their roles defined for them when they

begin is extremely valuable. We really only have a short time—about 16 weeks in a

semester—and that’s not enough time to get anything of significance done even in the

best circumstances. When you spend a few weeks just organizing and figuring out who is

going to do what, you’ve lost valuable time. The fact that roles are defined so that there is

minimal overlap helps a great deal. I believe that it would be true likewise for our TSP

groups in industry.

we’ve got a year. The students do part-time in the fall and spring. They do about 12 hours

a week. Then, they do full-time during the summer. Because of the pressure of the other

core courses and electives and just getting organized, I was getting late deliverables. In

many years, deliverables that were due in December were arriving in April on many

projects. This year, I know that four teams used TSP for the first time and at the end of

the second cycle, which would be somewhere in November, we had more things done

related to the project that we normally would have had in April. That impressed me. All

four teams were more productive than all of the teams prior to them, except for a couple

of really outstanding groups in the past that were just exceptions.

data, not just your own data, but a way of judging your own data by looking at the group

data. That team dynamic is an important component. Having said that, I have had

occasion to interview engineers in the field who were very nervous about anyone seeing

their data. I couldn’t find any reason, through interviews with other people in that

organization, that they should be nervous. Yet, they were nervous about sharing personal

data with teammates for fear that they would misinterpret the information. I’ve also come

upon engineers who have been using it for six months to a year and their first question to

me, when they hand me their data is, “Is this any good?” They’re not able to step back

and evaluate it on an objective basis, or at least they don’t feel confident in doing that.

So, having peers to interact with, who don’t threaten the protectiveness you might feel

over your data, is a very important part of successfully adopting the PSP. TSP is one of

the best mechanisms we can find to build in that peer interaction. You also have a coach

whom you might view as a mentor, someone of some seniority, whose opinion you

respect. Having that person to help you analyze your data is a very important component

to the continued use of PSP in industry.

valuable part: the self-analysis. I know there’s probably a fear that it’s used to judge the

effectiveness of individuals by management. I know that there are some groups in certain

organizations that try to dispel that. At Motorola, even though management has said that

they wouldn’t use it as a means of evaluating individual performance, they did want to

look at team PSP data. They decided that each team would elect a data security person

who would collect the data, roll it up, and answer questions about individual data without

associating it with individuals in a way that they could be identified. That is a fear and

it’s too bad because I think the data is very valuable for improving individual

performance and, in the long run, team performance as we figure out ways to get team

metrics. I’m not sure we have that yet.

methods for developing software weren’t getting diffused more readily. In your research,

you discovered that PSP is not getting diffused as readily as it might. What do you think

is the reason for that?

engineers are involved in the initial decision to adopt a new technology—for example, to

bring in PSP—has an impact on how widespread the use is. In general, we found that the

degree of developer involvement was on the low side. That could explain some of the

reasons why the use is not as widespread as we thought. In cases where the involvement

by the developer is greater, we tend to find a greater degree of use of PSP by that

developer.

Another issue is training. It turns out that training is an important indicator of sustained

use. So, the degree to which the developer was able to receive quality training and was

able to have sufficient time to work on the training-related assignments also seems to be

an important factor in the diffusion.

understand how to adapt it to a requirements stage and also to a maintenance stage. Some

people felt that their projects were primarily maintenance or the evolution of an existing

system. They weren’t quite sure how to adapt PSP to those types of environments.

Respondents indicated that PSP did seem to provide them with improvements in software

quality as far as decreasing the number of errors. However, we weren’t able to show

productivity improvements through the results of the survey. Respondents said there was

a lack of immediate improvement in development productivity. I think part of that could

be due to the fact that we are talking about an innovation, so there is a learning curve.

management persistence—the idea that management must not only support the effort, but

constantly encourage it.

in our research—we found somewhat equivocal results. It’s clear that champion support

is essential to get an innovation off the ground. But we also found a need for developer

involvement in the decisions to be made to bring that innovation in. Some people felt that

the champion was too demanding: “You will do this no matter what.” That took away

from the developer’s personal control or personal choice in the matter. One of our

recommendations in the report was that there is a fine line to be drawn between the

amount of champion support and the amount of user decision-making and involvement

on the developer’s side.

same time, you don’t want to do none at all. There’s that fine line. As far as management

support for training, we had good news. Software developers seemed to believe that

management had provided enough time and funds for training. They felt that they had

enough access to PSP experts and felt that they had a fairly good network of support.

There is, however, the challenge that managers give them enough time to go to class but

not enough time to effectively work through the assignments. There was also a comment

about class sizes: classes might be more effective if they’re smaller.

the persistence of the faculty throughout the curriculum. You can’t simply teach it in one

course and then expect students to acquire some sort of disciplined process that they’re

going to automatically apply in other areas.

who had to sell it up the management chain. It was brought in by the owner of the

company and pushed out, which has some obvious advantages. We changed a lot of our

management and organizational practices to be consistent with the use of PSP. We

developed software engineering policy statements, stating that these things were going to

be used. We integrated it into our performance-evaluation system at both the engineer

and project-manager levels. Project managers have to report quarterly to senior

management about which PSP practices are being applied to their projects. We provide

support via PSP user group meetings at least once a quarter. Our organizational goals and

objectives include developing high-quality components, which indirectly relates to using

PSP in our development work. We decided that this was important to us, and that we

were going to have to change our culture to accept it. We really took a serious look at it

and changed things at the organizational level to support it all the way.

totally different from when they write about something that they’ve thought about. It’s as

if they write with two different styles. What we’re trying to do with PSP is to convince

them that there’s really one style of work. You don’t have a formal working style and an

informal working style. Your approach to the work is disciplined all the time.

TSP?

thought I was a lot better software developer than I was. But it did show me ways to

improve and I saw improvement from the beginning. That’s the sort of experience you

hope all students and engineers who go through PSP training realize themselves. I have

also changed my view of the way I teach programming. Software development and

software engineering have changed dramatically. I hope I am much more of a coach and a

mentor and a supporter than I am just a lecturer or a one-way facilitator. When I was at

the SEI and sat through the engineers’ course, I saw people who had been programming

for 10 or 20 years who were struggling with some of these problems. The SEI does a very

good job in their afternoon labs of helping people plan and learn PSP and develop quality

software. I thought, “Here are some experienced engineers who are struggling with how

to develop programs effectively.” We just throw things at students sometimes—

especially at the undergraduate level—and expect them to develop their own effective

process. So, it changed my viewpoint of the way we’ve been educating our computer

scientists and software engineers.

impact on my associates and me. The industrial delivery of the course is a substantially

different professional training course than most of the courses you go to. This is not a 9-

to-3 type of course that meets for a handful of days for a light break away from the

normal workflow. This training is much more intense than what people are facing in their

normal daily efforts. There needs to be a strong understanding of the commitment and

time it takes to get through the training and an effort to make sure people have enough

time to get through the assignments and realize the full benefit of PSP. Then, afterwards,

it’s important for people to understand that since this is not your typical professional