The Elusive Search for Categories
DAVID CARNEY

One nagging issue about commercial off-the-shelf (COTS) products that surfaces with alarming regularity is category: As people debate whether to choose some software product, they often ask, "Is this product really COTS?" The assumption behind the question is that there exists some collection of attributes that define "real" COTS products, permitting us to categorize any particular commercial product. Implicitly, the goal is to segregate true COTS products from the false (and, presumably, less worthy) ones, and thereby better follow the spate of recent policy directives that affect us all.

If only we could. With few exceptions, however, it is remarkably difficult to determine, with any basis in objective reality, the hard boundary between what is "really a COTS product" and what is not. Even when we can make such a test, it seldom gives us the help we genuinely need.

 

Toward a Definition of COTS

There are a number of reasons for this unfortunate situation. One reason is that we can’t even find a consistently agreed-upon definition for either the "commercial" or the "off-the-shelf" parts of the acronym. So defining both together is a remarkably difficult task. For instance, a standard dictionary definition of "commercial" includes such characteristics as:

Of or pertaining to commerce ... involved in work intended for the mass market ... designating goods produced in large quantities for use by industry ... having profit as a chief aim ...1

This is probably sufficient for informal use. But as a guide for making distinctions in government acquisitions, this definition is full of holes. Is it truly sufficient to merely intend that your software product is aimed at a mass market? Is your software thus automatically "commercial"? Or if you produce some software "in large quantities"--surely the easiest of actions with computer software--is this enough to officially "be commercial"?

If you work for the Department of Defense, you don’t get much help from the Federal Acquisition Regulation (FAR), which offers its own definition of "commercial item." The definition is shrouded in Washingtonese, but the key points are similar:

Property ... sold, leased, or licensed, or offered for sale, lease, or license to the general public ... [or that] will be available [or for which] ... modifications are customarily available in the commercial marketplace or minor modifications are made to meet Federal Government requirements ... [or is] ... a nondevelopmental item developed exclusively at private expense and sold competitively to multiple state and local governments.

Again, the seeker of well-defined categories finds little to go on. "Minor modifications" could include a whole galaxy of changes, and once again there are no specifics about distinguishing the intent to sell something from actually having sold it.

As with "commercial," the same problem appears when we try to pin down what "off-the-shelf" means. Probably no one (no one who regularly reads this column, at least) would contend that, in the context of serious software systems, "off-the-shelf" should connote the way that we use simple computer games (i.e., take off the plastic wrap, pop the disk in, and off you go). But if it’s not that simple, users presumably have to take some minor individuating steps to make the software run in their own systems. And if the user has to do at least some diddling, where do we logically stop? As we move down the slippery slope from simple installation through more complex parameterization, through tailoring, through customization, to whatever comes next--when do we look at Toto and realize that we’re not in Kansas anymore, and that our software is no longer "off-the-shelf"?

The SEI, in developing a series of COTS-related courses, has provided a definition that seeks to fill some of these holes. It describes a COTS product as one that is:

• sold, leased, or licensed to the general public
• offered by a vendor trying to profit from it
• supported and evolved by the vendor, who retains the intellectual property rights
• available in multiple, identical copies
• used without modification of the internals

This is a distinct improvement, and this definition has proven valuable on many occasions. But it is also true that in many real-world cases, the products that people care about don't quite fit all of these criteria; a lot of things that people are considering for their systems still fall into the grey areas, and do not clearly fit this definition.

So I’ve come to the conclusion that the attempt to pin down whether a product fits the perfect definition of "COTS" is the wrong question: Even if you can determine absolutely that a given package is or isn’t COTS, that won’t tell you what you really need to know. In other words, something that’s "really COTS" isn’t always better, or cheaper, or faster; there’s simply too much context that’s needed.

 

How to Proceed?

I agree that we need some help in this cold, cruel world of commercial software. Like any marketplace, it is filled with pitfalls, and many struggling managers would welcome some guidance in navigating them. But rather than the simplistic acronym approach ("Is it or isn’t it COTS?"), I think that we all would be better served by making the effort to find some truly useful characterizations of commercial software. These could aid us both in making choices between competing alternatives, as well as in making the more basic "build vs. buy" decision.

Let’s start by turning things upside-down. Let’s see if we can characterize some commercial software not by what we’ll save from using it, but rather by what it will cost us.

One thing that immediately comes to mind is that commercial software implies process change: It is generally understood that "going commercial" means some kind of business-process reengineering. Conversely, if you absolutely must have an exact match for an existing process, then the commercial route is not for you, and you’d better go custom. Thus, there is some sort of correlation between how much you’re willing to change your business process and how much you can accept a commercial software product as-is.

This suggests that we can place the notion of business process--more precisely, the degree to which a business process must change--on a spectrum whose scale is bounded by a custom development at one end and a totally as-is solution on the other. The former end would imply no change at all to my processes, which is true (at least in theory) with custom code written entirely to specification. At the other end, where we take code from some other source, and with no changes, my business processes would undergo a lot of change, maybe even massive process reengineering. This "change-to-process" spectrum would look something like this:

Figure 1: Change-to-Process Spectrum

Clearly, we’re being a little simplistic here. On the left side, I’m asserting that a custom development would mean "no change at all," and on the right, I’m sidestepping what "significant reengineering" would really entail. Either of these assertions is arguable. Still, this overall notion--that the amount of process change is related to the amount of custom vs. commercial software--seems reasonable. Rather than worrying about whether some solution is or isn’t really a COTS product, it may be of more use to map its potential value (and its potential savings) against the cost that is implied by how much reengineering it will demand. That’s a better deciding factor than pedigree.

Let’s shift to an orthogonal view of the same question. It’s clearly beneficial to know, before we commit to a commercial product, how much we’re going to have to tinker with it. So I now posit a spectrum of the changes that we make to a product (either must make or choose to make); its scale is bounded by the degree of change that is possible. Thus, borrowing from my earlier example, we could move from the triviality of a computer game (pop in the disk with no change) through more and more complex code changes: installation, parameterization, tailoring, customization, finally arriving at some sort of gross internal modification. This "change-to-product" spectrum might look like this:

Figure 2: Change-to-Product Spectrum

This spectrum is a bit more ambiguous than the other one. For instance, up to a point, vendors of some software packages expect and encourage changes: Whether you’re installing a simple UNIX package or implementing Oracle Financials, there is a certain amount of work that the vendor expects you to do. It’s equally true that there are certain things the vendor doesn’t want you to do. Locating exactly where that distinction lies is sometimes difficult (though it’s a sure thing that when you violate it, you’re on the Dark Side). Complicating this is that the scale I suggested ("degree of change that is possible") is based on the visibility of the code and how easily we can access it. But "code" means a lot of different things these days: not only third-generation languages, but fourth-generation languages, schemas, graphical user interface (GUI) languages, and so forth.

Whatever else is true, there are a lot of products commonly considered "COTS" that appear all along this "change-to-product" spectrum. Knowing they’re "real COTS" is of trivial importance. Knowing how much work you have to do to use them is much more valuable.

These spectrums are only two simple possibilities, and I’m not arguing that they are sufficient to make an important decision about choosing some commercial software. I am arguing, though, that a commercial software decision is massively context dependent, and simply giving some product a label, even if one can, doesn’t address the problem. What the decision-maker needs is greater insight into the details of his or her particular context and more information about how candidate products fit in that context.

 

An Example

To realize just how context dependent a commercial decision is, we only have to populate the "change-to-process" spectrum with some real software. Take word processors: Where would we put Microsoft Word? Well, that depends. If our users are switching to automatic document generation for the first time (if, that is, typewriters still existed!), then it would be a huge process change, similar to an earthquake, and far over on the right side. Next, suppose our users currently do word processing, but only with standard generalized markup language (SGML). This change would still be an earthquake, but lower on the Richter scale. Suppose our users are currently using FrameMaker. In that case, while there are many details that change, the essential process is very similar. So even in deciding about Microsoft Word, which is unquestionably a "real COTS solution," we still have different degrees of process reengineering, depending on the current state of the business process.

It might be interesting, at some point in the future, to superimpose these two axes into an X-Y graph, and populate it with other actual examples. But I’ve probably gone on long enough, so I’ll leave that as an exercise for the reader.

Finally, I don’t think that there’s anything revolutionary in what I’ve argued for. Everyone knows, deep down, that labels are at best guideposts--"What’s in a name?" says the adage. Adages aside, however, and whatever we all may know deep down, we’re all still suckers for simple categories, pedigrees, and so forth. It’s easier. It’s less effort. My argument is only that, with commercial software at least, the adage is true; and fitting a product into the category of real COTS seldom does us much good.

So I’ll sign off for this issue. Next time, I’ll discuss the topic that I promised for this issue--namely, some relationships between commercial software and risk management. Stay tuned.

 

 

 

 

About the Author

David Carney is a member of the technical staff in the Dynamic Systems Program at the SEI. Before coming to the SEI, he was on the staff of the Institute for Defense Analysis in Alexandria, Va., where he worked with the Software Technology for Adaptable, Reliable Systems program and with the NATO Special Working Group on Ada Programming Support Environment. Before that, he was employed at Intermetrics, Inc., where he worked on the Ada Integrated Environment project.

 

The views expressed in this article are the author's only and do not represent directly or imply any official position or view of the Software Engineering Institute or Carnegie Mellon University. This article is intended to stimulate further discussion about this topic.