the different perspectives of our technical initiatives. How can we improve requirements

engineering as part of our mission to improve software engineering practices and

technology?

the problems that we face in our work with COTS (commercial off-the-shelf)-based

systems stem from requirements--this belief that you can do things the way you used to

do them. I just heard someone say yesterday, “We had this great success in setting

requirements and in developing this particular system.” Their “success” was that they got

the system fielded. It’s an appalling system--it’s unmaintainable, it can’t be upgraded,

and it’s going to be a disaster. But they’re ready to do the same thing again because

they’ve had a success!

You almost wish that the system had not been fielded. But I think if we want to be

software engineers, we have to consider the entire cycle, the entire life of our systems

from inception to final turnoff, cradle to grave. And that involves figuring out what that

system should be, which is collecting and eliciting requirements and maintaining those

requirements throughout the life of the system. Because if you just do it once and then

throw it away and let the system evolve, you’re in trouble.

customers either have nothing in the way of product line requirements and are looking to

establish some basic set of requirements--they aren’t really looking in detail at how to

elicit, manage, or track requirements--or they come to us with an architecture already in

hand and say, “Now what do we do with it?” Or they come to us with a system that’s in

trouble and say, “How do we reengineer it?” So it doesn’t seem like people are coming to

us saying that they need help with managing requirements. We might say that there’s a

problem there because they can’t track the requirements. And they say, “Well, give me a

tool to do it.”

and asking us to analyze their existing systems to see where the security holes are. Right

now we’re really trying to push the idea of having them look at their architectures up

front to see if they’re survivable. There is not as much awareness of these up-front

survivability issues as there should be, but the awareness is growing.

broader systems perspective--to be able to see the whole as more than the sum of the

parts, to be able to assess the behavior of the system as an integrated entity. Complex

feedback effects are very difficult to understand and often counter intuitive. As system

complexity increases, it is more and more difficult to predict, at the requirements phase,

what exactly will happen. As a result there is requirements churning, delaying project

deadlines and pushing up costs.

I believe this situation can be addressed with the appropriate use of simulation

techniques. Simulation has been used for many years within the engineering community

to predict everything from traffic flow to aircraft wing loadings to the likelihood of a

reactor accident. However, the software community has, ironically, been slow in adopting

this technology. I don’t believe that simulation is a silver bullet, but it is more than a

technology in the sense that its use can raise awareness with respect to the importance of

dynamic effects in complex software systems. By using simulation to support

requirements, a customer can achieve a much-improved sense that a supplier understands

the issues. In addition, if changes are proposed, a simulation of the changes is likely to

wring out potentially unforeseen consequences.

Simulation software is now mature and readily available, but it will probably take a few

dramatic examples of simulation supporting software requirements before the community

takes notice.

customer coming in and saying, “I’ve got this legacy system, it’s ancient, we have to

build a new one, how do we do it?” And the first thing is for them to even understand

what it is they have in their existing system. They have documents that are hundreds of

pages long that are supposed to describe their requirements, but they don’t. They’re a

mishmash of operational procedure, of detail, of flowcharts...but the documents don’t

provide any understanding of what their requirements are.

mandate to use COTS products, one way or the other. And so they’re trying to figure out

how to do it. We tend to work on the design and the technology, but one of their big

problems is understanding the requirements that they need to build to.

some of the downstream problems we deal with, and we don’t have nice, neat solutions to

hand to people for problems related to requirements. We can’t say “If you go off and use

this analysis technique or that elicitation technique, everything is going to be fine.”

problem too.

stuff. It’s fundamental communications, not writing things down in a mathematical

formula. It’s going out there and talking to people, and eliciting what the requirements

are, then capturing them in some way that an MBA can understand.

had a little lamb,” and you see that the meaning changes depending upon which word you

emphasize. So we have to look at the whole idea of the way that we communicate, which

is really an interpersonal skills issue.

There are folks out there who are doing good work. Barry Boehm, for example, with his

win-win stuff [Boehm 99], is in my opinion one of the folks who is at the forefront of this

kind of activity. All of us have to do requirements elicitation and analysis for our

particular projects. But in terms of looking at requirements engineering as a discipline, I

think you really need somebody or a group of people who focus more on the elicitation

side and the capturing of requirements, which is a communications issue more than it is a

formality issue.

issue, but you jump to the conclusion that writing things down mathematically is not the

right thing to do; I think this is a bit of a leap. I was involved in an exercise with the IEEE

POSIX.21 standard for realtime distributed systems communication. We communicated

in English to the people who were developing the requirements about the flaws that they

had in their existing requirements set, but that communication was based on a

mathematical analysis. I agree with you wholeheartedly that I would not take 20 pages of

mathematics--or even one page--to the average system acquisitions individual and say,

“Here, now you know what the system is.” But you can make the interpretation back into

the system’s own language or domain of discourse because you can see from the

mathematical logic the consequences of your decisions; I will tell you what those

consequences are not in a stream of symbols, but in a stream of words that have meaning

to you, and you can tell me if I’m right or wrong. And if I’m wrong, I can adjust my

model, and if I’m right, you can adjust your requirements.

The point I was making is that a lot of the work I see that is reported under the

requirements engineering label is really dealing with a requirements specification

language, a formal methods kind of approach: “We’re going to have a complete,

consistent, unambiguous set of requirements.” And then you go out and you try and

explain that, because when you try to translate from the formal specification back to the

English, you inject ambiguity back in. That’s the reason you went to the formal

specification in the first place.

You can get fairly detailed with a rigorous analysis, but when you try to abstract that

back up to the CEO or the MBA or whomever your customer is, things you have captured

absolutely precisely are just not what they really meant. Or as we’ve said several times,

they don’t know what they really mean, but they’ll figure it out as we iterate with them

on what the system should be.

to use formalism and pragmatic, focused models as communication/understanding

vehicles. As people analyze requirements specifications, they create an abstracted model,

and there is communication. They begin to understand the engineer who is building the

system, and they also make users, clients, and other stakeholders aware of the issues that

emerge. So we see that using formal language effectively as a communication as well as

an analysis vehicle is a very effective way to identify errors in requirements and make

them explicit. People may consciously decide about a certain requirement, for example,

“I’m not sure about this, I’d like to defer it.”

process of building the model--of abstracting out the essential elements--that helps focus

discussion about a system and also identifies ambiguities.

you the engineer. It doesn’t address the communication from you the engineer back to the

people so that you can fix what’s in their heads because they’ve got it wrong.

then you can communicate in a way that they will understand. If you’re talking to a

software programmer, and you say, “Now here is the language specification for C++,”

you might use the formal BNF specification to help the programmer understand the

syntax for the language. But if I were trying to teach some manager how to do

Programming 101 so he could write a little sort program, and I put one of those

specifications down, I can predict that there would be some resistance to that form of

communication.

graduate students a formal specification, plus a natural-language discussion, for their first

exercise. In fact, for all of their programming careers, students used formal specifications

in their exercises. And the university found that successful--that’s why they did it.

for me. I guess that natural-language discussion is critical: the better you are at that, the

more success you’re going to have.

invest in both. They want to pick one or the other, and I’m not sure that one or the other

can do the job that needs to be done.

requirements. Usually formal requirements are applied to narrowly focused elements of

the problem while natural-language requirements complement this in the broader scope.

For high-reliability and safety-critical systems, formal requirements make a lot of sense

for precisely describing the system's interactions, but that's not necessarily best for

providing an understanding of overall capability.

surrounds it. Any mathematical model can state very precisely what the requirements are,

but there’s no intuition. And you can provide all the intuition about that precision in

natural language, so that it’s not just 20 pages of mathematics. It’s mathematics plus text

that says what the mathematics mean.

in structural engineering. They go to mathematics very quickly, they go to formal

representations, and they’re integrated--mathematics are part of how they define

requirements, part of how they accomplish this communication and translation to the

engineering staff and to the other people involved. I don’t think there is anything

fundamentally different about building software than building anything else, but there’s a

larger focus on intellectual intangibles up front that are more difficult to describe.

functions, many times more complicated than any other manufactured artifacts. Even

well-understood programs have incredible complexity. In a previous roundtable

David Carney likened COTS software to bridges as opposed to screws; however, I think

that he’s doing software an injustice--it’s more like building a country’s road system

when the pieces we have to interconnect are the towns! This complexity takes software

out of the realm of the other engineering disciplines.

A second significant difference is that software is, by its nature, of extreme generality. A

toaster makes toast and could, if you bought an expensive enough toaster, be coupled to

all sorts of other household artifacts so that your morning toast was available, say, 10

minutes after your morning shower--whenever that occurred. That said, it’s still a toaster,

and if I wished I could create the mathematical language to describe and engineer

toasters.

The mathematics of bridges are well understood and relatively simple. However, for

software, since it has no defined purpose, the best we can do is create a mathematics

capable of modeling any aspect of software. Unfortunately, this leads us to all

computable functions (a technical term, not to be confused with computation)--this is a

lot of mathematics to cover and means that we are without a domain-specific language to

use for our engineering (unlike bridges that deal in tensile strength as well as forces and

moments).

Another point to bear in mind is that software development is really new; we’ve only

been developing what we consider to be software for a tiny fraction of the time during

which we’ve been building bridges.

that, in traditional engineering, the results of what you produce are mostly tangible.

Buildings, bridges, and aircraft all have a physical presence that software does not. I

think this physical presence allows one to more easily visualize the issues and to think in

metaphors. For example if I understand the reason why bridges are structurally sound, I

can probably use that knowledge in designing safe buildings--or even aircraft. I'm not

sure that this intellectual reuse is as portable with software. At least it is not being used

that way today.

very general observation that other engineering disciplines rely heavily on mathematics

as a basis for describing and understanding their systems. I did not mean to select

structural engineering specifically. I believe this is true across the board in engineering

endeavors.

My point is simply that other engineering disciplines rely on mathematics (formalism)

and so, if software engineering is to be an engineering discipline, it too should include

mathematical formalism as integral to doing “good” engineering for software systems.

The issue is not whether formalism should be included but rather how it should become

part of the discipline.

Also, I was speculating on what differentiates software from the other engineering

disciplines. While it may seem that software engineering does not have much in common

with mechanical engineering, I think it does. But consider another example--electrical

engineers (digital) designing microprocessors or custom application-specific integrated

circuits. I believe the issues and problems here closely align with those of software

implementation. It can be argued, though, that the similarity is at the

design/implementation levels rather than at requirements. But in looking at requirements,

one may ask how software requirements engineering differs from systems requirements

engineering.

Pat, relating to the complexity premise, I am not sure about this perspective. Consider the

Boeing 777. I believe that the complexity of the entire aircraft is greater than the

complexity of the software that comprises only part of it, and that the requirements

specification for the software can be considered as resulting from doing the system

design for the aircraft.

something tangible to look at. So when you’re saying “I want to build a bridge,” if I’m

the mayor of Pittsburgh, I can say “I want a bridge to go from here to over there and I

want to be able to run 50,000 cars every day over it.” And so you can specify from the

user’s perspective.

Dave is absolutely right, when the engineers started doing that, they started getting into

the mathematics from Day 1, but those mathematics are communicated to the city

engineering office, they’re not communicated to the mayor. There are different audiences

for the different ways that you capture things, and the problem that we run into in the

software world is not that we don’t have different audiences, it is that we try to

communicate the same way to all of them.

the later design reviews was a review of the user interface and displays. At that review,

we found out that these folks who had sat through all the previous reviews had no

understanding of what was going on, because they were looking for data on the displays

that wasn’t being computed in the system; and if they had understood everything that had

come before then, they would have realized that not only was the data not being

displayed, it wasn’t there, it didn’t exist. Because they didn’t understand the methods that

we were using to present our designs, they were unable to internalize that until they

actually saw the user interface; that was the only way they could comprehend what it was

that we were trying to do.

scenarios are popular because it’s in the actual use that people internalize what it is that

they’re getting. Rapid prototyping is another technique--all of these techniques are ways

of getting the user wrapped up into the system.

descriptions of requirements that evolve should be characterized in terms of the

environment so that they become more real. The entities that are identified need not be

tangible, but they must be real in terms of the user, and I think it’s important to keep a

balance between what’s real on the one hand and the need to consider COTS products on

the other. Because COTS products will constrain the designer and ultimately perhaps the

performance of the system. This is not explicitly communication, but it’s really wrestling

with ideas that have traditionally been discussed in terms of the “what” and the “why.”

Use cases, scenarios, and other similar strategies enable people to describe the system

and how it works within the environment.

This person’s organization had developed, I think, 156 use cases to describe its system.

The use cases were stated in terms of processes that the system had to go through. We

were sitting there with the person who had developed all the requirements, and also

across the table were all of the program managers. He said “What I would like to do is hit

on one of these functions and see what use cases apply.” And the guy’s jaw kind of

dropped, because there was no mapping between the use cases, which were process

oriented--how to do tracking, analysis, engagement planning--and the functions that were

actually built into the system.

software and the supplier of that software to be geographically distributed. This usually

means that they have to get together on a periodic basis to “sync up.” During the

formative stages of a project, when requirements are being developed, this interaction can

be particularly critical, and interactions should be most frequent. However, the

impediment of distance means that such interactions do not take place as frequently as

they should, with the result that communications quality can be degraded. Phone and

email communications just do not measure up when high bandwidth interactions are

required. Clearly the quality of the requirements lays a foundation for the quality of the

resulting software system, so there should be a great motivation to really work hard at

effective means to communicate easily early during the early stages of such projects.

This is why I believe that collaboration technologies should be used to help groups

develop requirements when these groups are geographically separated. These

technologies are rapidly maturing and coming down in cost, so it is becoming

increasingly appealing to use them. They allow for a wide variety of types of interactions

among individuals, and the automatic capture of session interactions provides a historical

record of activities.

software engineers manage these different levels and determine the “real” requirements

of their systems?

documents, that they call “functional specifications,” and these are all in terms of user

functionality, such as “The user types these two letters and these two digits, and this is

what will happen.” So in essence, they are the requirements for the system, and that

document is maintained and is kept up to date. And, given the life-threatening nature of

their systems, people do not make changes without good reason. But is that document

really a “requirements document?” My guess is that it is not, at least at the level of “What

are the requirements of the system?”

The requirements of the system are that it be able to maintain separation of aircraft, to

maintain the safety of people as they’re flying en route and as they’re landing; it’s not

that the controllers shall be able to type these two letters and these two digits and achieve

this effect. I think that people lose sight of the real requirements, and they don’t see the

value of investing the effort in maintaining currency between whatever the requirements

document is and the code. Because while the design documentation and the architecture

documentation may never change, they know they’re going to change the code. So there

is a tendency to view everything else as overhead. That’s why you get the situation that

we’ve all seen with systems that evolve, where the initial requirements document

becomes irrelevant. We do not recognize that the requirements are recorded in a living

document that must be maintained and that must survive and evolve throughout the life of

the system. We just don’t do that, because it’s expensive and there’s no return on it; it

doesn’t get another byte of code written.

keeping the system in operation much longer than was intended, accuracy was lost, and

hence the anti-missiles didn’t go in the right direction. The change was a procedural one--

just take it down and re-initialize it periodically.

problem. After a couple of weeks of continual operation, they’d run out of virtual

memory. Procedurally, you needed to reboot the system. But if you’re running on a 24-

hour-a-day, 7-day-a-week system, you do not want to reboot that system. Yes, you can

get around these things, but I see confusion in the fact that people have these operational

procedures documents but don’t realize that there are also requirements on the overall

system. We think of requirements that are system requirements or software requirements,

but we don’t consider that the man in the loop and other environmental factors really

form this thing that is the system.

the user manual or the operator’s manual be considered the real requirements

specification for the system? For example, in the case of the FAA and the need for

separation of aircraft, a requirement might be that there should be an alarm that goes off

or some kind of blinking light that says these craft are getting too close together. Or the

procedure for operating the system should say that when these two aircraft get too close

together you should tell them to go in different directions or something. I know that there

are non-technical requirements in there, but one of the things that I’ve encouraged folks

to do is to make sure that their user manuals and what the code actually does are

consistent with one another.

requirements, and X number of ‘shalls’ in our document,” and this is what you have to

address. At some level, yes, you have to have that, that forms a contract. But is that

something that’s manageable or maintainable, or even something that I can elicit from

people?

is, in essence, invisible to the user? A lot of times when we get into functional

requirements, they’re actually design statements, they’re not really requirements. It’s the

requirements about what the system will do, what the user can expect to achieve, which I

would hope would be captured in some kind of user manual or something like that.

user, that may impinge upon the design. For example, if the FAA writes requirements

only in terms of things that are visible to the user, they will have many different types of

maintenance to do. If they can require that everybody runs on this operating system or

uses that hardware, which is completely invisible to the user, they can save themselves a

whole bunch of maintenance.

what environment you’re from, a lot of times there’s a maintenance document that is part

of the deliverable set, which includes the user manual, the operations manual, the

maintenance manual...

maintain the operating system. Periodically you will have to do operating system

upgrades, but you don’t maintain the operating system. I think that you would rather have

one operating system and one set of procedures for how to reinstall that operating system

than to have 20.

all of the operational procedures tell you what to do with aircraft with the assumption that

you’re tracking them correctly. But there’s really a requirement of the system that you

track all aircraft, and that might not appear in the user manual. And all of the software

that’s underneath it isn’t captured in the statement that you’re going to display.

have available. But there are other requirements for completeness and currency of that

information that probably don’t appear.

thing right. Many if not most of the errors that occur downstream, even in code, were

requirements errors. So there are two elements of getting requirements right. One is

something we’ve been talking about--eliciting them and capturing the real needs. Once

that’s accomplished and we have in fact described real needs, are they consistent and

correct in and of themselves? That’s the kind of analysis that we’re focusing on. Are they

complete, consistent, do they make sense, do they really accomplish the objectives that

the stakeholders wanted? These questions must be addressed at the requirements level,

especially given the data that shows that the cost of correcting an error that occurs

downstream that was originally injected in the requirements is quite significant.

We’ve all observed that, over time, the code changes, the design changes, and the

requirements change. So the point that Dave is making is absolutely valid, but the

problem is that if we had a stable set of requirements that was unchanging, then the more

rigorous techniques would be much easier to use. But when you have a dynamically

changing environment, how do you deal with the consistency and make sure that

everything is kept up to date? It’s a human behavior issue as much as anything else.

language or natural language. But a potential solution is to capture as many of the

requirements as appropriate and can be effectively accomplished in a formal language

and then allow automation to handle a lot of the application-independent things that are

problematic and that require a lot of time but not a lot of intellectual activity on the part

of humans. There are solutions that are tending in that direction, so that you have a line

from requirements all the way through to code, and the ability to then do checking and

analysis.

been on the requirements phase. Even when they talk about analysis, they’re talking

about something that is much closer to design than it is to requirements analysis. The

approach starts with the assumption that the requirements already exist, and elicitation is

not the emphasis.

In the product line area, we tend to apply object-oriented analysis to the up-front

elicitation; but again, it’s in a rather narrowly focused area within a bigger problem. And

once we give our results back to the client, there’s a tendency to look at architecture

development activities, and then never to return back to the requirements as they exist in

that up-front analysis effort. The requirements will be evolved through the architectures,

through change scenarios, or some other approach, which lets clients explore what

they’ve got in hand, what kinds of changes the system is likely to undergo in the future,

and to refine with those things in mind. They’re not going to go back and elicit further

requirements through a formal elicitation or management process. And I think that’s

likely to cause a problem downstream. At the end of implementation or even design, the

clients don’t really know what requirements they have.

This is similar to what Pat said earlier--a lot of people come in with these legacy systems

and ask “What do we have?” So one of the things we can take back to other customers is

to say, our experience has shown that, without formally tracking and managing

requirements effectively, you’re going to run into some of the same problems

downstream as those you’ve encountered in the past. They may hit later than they do

now, but they’re still going to hit, and they’re going to probably be as severe if not more

so, because systems are bigger and more complex, and they affect more people than the

old standalone systems used to.

The situation we see commonly is that people do some up-front analysis, and they have

some elicitation and some requirements, but as they get into the architectural design, they

never go back and update the requirements. So within months after the first set of

requirements is complete, the requirements have already been overcome by the

architecture, and they are never re-examined. To these people, requirements elicitation

and management aren’t even issues. The issues now are management and evolution of the

architecture. There may have been some requirements notions up front, but now they’re

embodied in the implementation.

lose the sense of integrity of the architecture, and going back further, the set of

requirements, and they start changing things in ways that they don’t reflect on. They

don’t go back and say “Does this match the original vision for this?” Sometimes the

vision isn’t documented or expressed really well, and so you don’t really know what the

impact is. But many times, people just don’t think about it; they think about the

immediate problem at hand and not the impact that it has on the original vision of the

system, which is why you see systems that become a mishmash even if they had a pretty

decent set of requirements or a decent architecture to begin with.