Advanced
One of the latest trends in systems development is to make greater
use of commercial-off-the-shelf (COTS)
products. While this change has been encouraged for many years for
all kinds of systems development, especially in the Department of
Defense (DoD), it is only in the early 1990s that the practice has
been mandated by everyone from industry executives to Congress.
At the same time, an open systems
approach to develop systems has been gaining popularity, with visions
of open systems that are "plug-and-play"
compatible, where components from one supplier can be easily replaced
by those from another supplier. Advocates of open systems often
confuse them with the use of COTS products, making it difficult for
the average engineer to know just what (s)he should be doing to
develop (and maintain) systems more effectively.
These two concepts- the use of COTS
products and the creation of open systems- are closely related and
complementary, although definitely not synonymous. The
purpose of this technology description is to
- define/clarify what each is
- explain the differences between them
- examine the benefits each brings to the development,
maintenance, and evolution of systems
A brief summary of the key points in this technology description
follows:
- COTS products hold the potential for cost-effective
acquisition of components and advancing technology, but they are
not necessarily open.
- Open systems emphasize (1) the use of interface
standards and (2) the use of
implementations that conform to those standard interfaces. Open
systems provide stability and a framework for the effective use of
COTS products and other non-developmental items (NDI) through the
use of interface standards. Well-chosen interface standards
can outlast any particular product, vendor, or technology.
- It is possible to use COTS products without creating an open
system; similarly, it is possible to create an open system without
significant use of COTS products or NDI.
- COTS products and an open systems approach are both means to
important system goals of improving the quality and performance of
our systems, developing them more quickly, and sustaining them
more cost-effectively. The greatest advantage can be gained from
using these two approaches together.
For further detail on COTS, open systems, and component-based
software development approaches, see Component-Based
Software Development/COTS Integration.
COTS. The term "COTS" is meant to refer to things that one can
buy, ready-made, from some manufacturer's virtual store
shelf (e.g., through a catalogue or from a price list). It carries
with it a sense of getting, at a reasonable cost, something that
already does the job. It replaces the nightmares of developing unique
system components with the promises of fast, efficient acquisition of
cheap (or at least cheaper) component implementations.
Because of the need for precision in procurement, the federal
government has defined the term "commercial item." The full text of
this definition can be found in the Federal
Acquisition Regulations (FARs); the following is a summary
[FAR
96]:
A commercial item is
- property1
customarily used for nongovernmental purposes and has been sold,
leased, or licensed (or offered for sale, lease or license) to the
general public;
- any item evolved from an item in (1) through advances in
technology and is not yet available commercially but will be
available in time to satisfy the requirement;
- any item that would satisfy (1) or (2) but for modifications
customarily available in the commercial marketplace or minor
modifications made to meet Federal Government requirements;
- any combination of items meeting (1) - (3) above;
- services for installation, maintenance, repair, training, etc.
if such services are procured for support of an item in (1), (2),
or (3) above, as offered to the public or provided by the same
work force as supports the general public, or other services sold
competitively in the commercial marketplace;
- a nondevelopmental item developed exclusively at private
expense and sold competitively to multiple state and local
governments.
Most people would agree that these ideas approximate the meaning
of "commercial-off the-shelf" (COTS) products, although the inclusion
of "services" as "COTS" is not often included. The salient
characteristics of a COTS product are
- it exists a priori
- it is available to the general public
- it can be bought (or leased or licensed)
Non-developmental item. A closely-related term
that is often heard in government (especially DoD) circles is
nondevelopmental item (NDI). In summary, a
nondevelopmental item is [FAR
96]:
- any previously developed item used exclusively for government
purposes by a federal, state, or local agency or government or by
a foreign government that has a mutual defense agreement with the
U.S.;
- any item described in (1) above that requires only minor
modification or modifications normally available in the commercial
marketplace to meet requirements;
- any item being produced that does not meet (1) or (2) above
only because it is not yet in use.
The key point here is that NDI refers to something that was
developed by someone else. It might have been developed by a
commercial interest, but typically it will have been for some other
government, department, or agency. A large-scale example of an NDI
would be a fighter aircraft that was developed by some other nation.
A more meaningful example in the current context would be a
navigation software subsystem developed for one aircraft that is
available for use in another aircraft. The salient characteristics of
a nondevelopmental item are the following:
- it exists, although not necessarily off some vendor's
"shelf."
- it is available, although not necessarily to the general
public.
- it can be obtained for use, although more likely off an
existing contract than off a published price list.
While there are certain reasons for using caution in applying the
definitions of COTS and NDI (e.g., how safe is a "minor
modification," and what if it just looks like a vendor has a
product, whereas it is in reality just vaporware?), they do fairly
characterize the features that are of interest to those who believe
that "buying COTS" is desirable and beneficial. However, although
closely related, there are differences between NDI and COTS
items:
- COTS products would most likely be found in some sort of
catalogue or price list, whereas it may be more difficult to
discover the existence of NDI.
- The range of possibilities opened up by NDI is broader than
what COTS products alone can offer, but NDI could lack the
commercial leverage that is sought in the use of COTS
products.
Open systems. The basic tenet of open systems is
the use of interface standards in the
engineering and design of systems, coupled with the use of
implementations (preferably, but not necessarily, COTS and
non-developmental items (NDI)) that conform to those interface
standards. This provides a stable basis on which to make decisions
about the system, particularly with regard to its evolution.
An open systems approach has the potential to reduce the
developmental cost and schedule of systems
while maintaining or even improving performance. The dependence on
stable interface standards makes open systems more adaptable to
advances in technology and changes in the marketplace.
When people use the phrase open systems, they most often have in
mind a system that is flexible and adaptive, one that is "open" to
inclusion of many products from many sources. The phrase open systems
often carries with it an image of easy "plug-and-play" between
components and products that were not necessarily originally
designed to work together. Open systems also hold out the promise of
being able to take immediate advantage of new technology as it
emerges, because it should be easier to plug in new technology,
either in place of an old component(s) or as a new extension of the
system.
Many different definitions of open system have been offered in the
past. To find a truly workable one, we must look more closely at what
it takes to make this vision a reality. For the purposes of this
technology description, open systems is defined as follows
[Meyers
97]:
An open system is
a collection of interacting software, hardware, and human
components, designed to satisfy stated needs, with the interface
specification of components
- fully defined
- available to the public
- maintained according to group consensus, and
in which the implementations of components are conformant to the
specification. One key part of the definition addresses a set of
criteria for the interface specifications/standards. Not only must
they be fully defined, but they must also be available to the public.
This implies that cost and public access may not be prohibitive
constraining factors; that is, the specification cannot be available
only to a selected group of people who have some special interest.
Anyone is free to obtain a copy of the specification (perhaps at the
cost of duplication and distribution, perhaps even at the cost of a
small license fee) and they are also free to produce and sell
implementations of that specification. It is also very important that
the specification is of interest to a wide range of parties and is
not exclusively under the control of any single vendor. To this end,
the definition includes the idea that maintenance of the
specification is by group consensus. Taken together, these criteria
come very close to requiring that the interface specification be a
"standard."
The main benefit of this definition of open system is that it is
operational. That is, it can be applied to a single system
at a given point in time. In contrast, most other popular definitions
identify desirable system qualities that open systems are expected to
display, such as portability, interoperability, and scalability.
Unfortunately, there is no way to measure a system with respect to
these qualities at a single point in time (e.g., "Portable" to what
platforms? And how many? "Interoperable" with what other systems or
components? And how many? "Scalable" for what use? To what size?).
Each of these qualities implies a relationship, either between the
subject system and some other unspecified one(s) or between the
subject system and itself over time.
This definition also supports the vision of what people hope to
achieve with open systems. The very phrase "plug-and-play" brings to
mind children's toys like Tinker
ToysTM and
LegosTM. The key to them is a small
set of well-defined, consistently-enforced interfaces. It also
invokes the images of hardware components that can be plugged
together because, for example, the pins and configuration of the
female connector are perfectly complementary to those of the male
connector. All these schemes have interface standards in common.
Most of the interface standards used in computer-based systems are
for application program interfaces (APIs), data
formats, or protocols. For all of these kinds
of interface standards, one can find fully-defined specifications;
without such clear definition in the specifications, wide variation
quickly emerges among implementations, and this undermines the
intended compatibility. Interface standards are made widely available
to the public to generate a thriving market for components that can
be plugged together. They are maintained using many forms of group
consensus; this precludes one vendor or group from making arbitrary
changes to the interface standard that will limit competition and
availability of alternative products.
Finally, for many of these interface standards it is possible to
tell whether or not a given implementation really matches the
specification; this is called
conformance. If the implementations
all match the specification/standard closely enough, then one kind of
incompatibility between components can be reduced if not eliminated,
and it may be possible to "plug" them into a system and get
them to "play" with the other components.2
On the other hand, if implementations only loosely implement the
standard or if incompatible interpretations cannot be detected before
trying to integrate a component into the system, then it is less
likely that the envisioned flexibility and adaptability can be
realized.
It is important to realize that it is possible to create an open
system, based on interface standards, in which no COTS products or
NDI are used. This might be necessary in a situation where, for
example, no COTS product conforming to the interface standard also
meet other system requirements, such as for real-time performance or
security. Although one would not gain the economic and schedule
advantages of using a component implementation that already existed
and was shared and supported by a number of users, the interface
standards would still provide the framework for future evolution of
the system (provided vendors do eventually pick up the standard and
produce conformant products). Potentially some future product may
emerge that does meet all the requirements. In the mean time, the
system enjoys the clarity and stability of a well-defined
specification.
There currently is a very strong push within the federal
government, particularly DoD, to make more use of COTS products and
NDI.3 In
addition to action by DoD leaders, the Federal
Acquisition Streamlining Acts of 1994 and 1995 directed the increased
use of commercial items, coupled with several adjustments to the
federal procurement regulations to encourage the new approach. Carney
outlines current government trends toward using
commercial-off-the-shelf (COTS) products [Carney
97a, Carney
97b].
The reasoning behind these directives and laws is that government
organizations typically spend far too much effort on defining to the
lowest level of detail the desired characteristics of systems and how
the contractors are to build those systems to achieve those
characteristics. Thus a lot of resources are expended developing
systems and components that often already exist- or exist in "good
enough" form with nearly the same capabilities- elsewhere. The
prevailing, and time-consuming, approach is always to develop from
the ground up; this approach results in unique systems each time. The
result is systems that are
- very expensive, with only one customer to bear the development
and maintenance costs over the life of the component or
system
- inflexible and unable to easily capitalize on advances in
technology
- historically fielding technology that is in excess of ten
years old
Shifting to a paradigm in which systems are built primarily of
components that are available commercially offers the opportunity to
lower costs by sharing them with other users, thus amortizing them
over a larger population, while taking advantage of the investments
that industry is putting into the development of new
technologies.
Open systems can have a positive impact either on new systems
development or in the context of legacy
systems. Although there is generally more decision-making freedom in
the case of a new development, open systems can nevertheless help
shape an evolutionary path for a legacy system that will help turn it
into a more flexible and maintainable system.
Many initiatives are under way, both in the DoD and in individual
services, agencies, and companies, that are designed to promote the
use of an open systems approach and to secure even greater benefits
than can be realized from the use of COTS products alone. These
initiatives are occurring because projects have been learning the
hard way that "just buying COTS" does not necessarily secure all of
the benefits desired. There are other problems and sources of risk
introduced by the use of COTS products.
COTS products are not necessarily open. That is, they do not
necessarily conform to any recognized interface standards. Thus it is
possible (in fact, likely) that using a COTS product commits the user
to proprietary interfaces and
solutions that are not common with any other product, component, or
system. If the sole objective is the ability to capture new
technology more cheaply, then the use of COTS products that are not
open will do. But when one considers the future of such a system, the
disadvantages of this approach become apparent. Many DoD systems have
a 30- to 50-year lifetime, while the average COTS component is
upgraded every 6 to 12 months and new technology appears on the scene
about every 18 to 24 months. Thus any money that is saved by
procuring a COTS product with proprietary interfaces will quickly be
lost in maintenance as products and interfaces change- the ability to
migrate cost-effectively to other products and other technologies in
the future will have been lost.
Even if the expected lifetime of a system is only 5 to 10 years,
the fluctuations in COTS products and technology result in a state of
constant change for any system employing them. Interface standards
provide a source of stability in the midst of all this. Without such
standards every change in the marketplace can impose an unanticipated
and unpredictable change to systems that use products found in the
marketplace. This situation is particularly painful when the vendor
stops supporting the product or goes out of business altogether, thus
forcing a change to a different product or vendor.
Program managers and lead engineers should also know that the
depth of understanding and technical and management skills required
on a project team is not necessarily diminished or decreased because
of the use of COTS or open systems. Arguably, the skills and
understanding needed increase because of the potential complexity of
integration issues, the need to seriously consider longer term system
evolution as part of initial development, and the need to make
informed decisions about which products and standards are best.
Paradoxically, given the desire to produce systems more quickly,
the emphasis on standards can actually be something of an inhibitor.
Some standards efforts, in their desire to achieve maximum consensus,
have very long cycle times (five or more years), which certainly do
not fit well with product development and release cycles. This
conflict is of concern and is being addressed by some standards
bodies, but it has led some projects to become involved with
consortia standards and also with de facto industry standards. While
these are often practical alternatives, they do have attendant risks;
the de facto standards may be proprietary, for example, and this
limits long-term evolution. The key is to weigh the risks of straying
from the three basic criteria (fully-defined, available to the
public, and maintained according to group consensus) against what is
gained over the long term.
The open systems concept has been at least partially introduced
into C3I systems, but it has been difficult to move into the realm of
real-time embedded systems, particularly
weapon systems, where it is much more difficult to find standards
that meet a system's requirements. Examples might include cases of
extreme real-time performance or security concerns.
There is limited documented experience with the open systems
approach. An example of successful use in the DoD is the Intelligence
and Electronics Warfare Common Sensor (IEWCS) program [IEWCS
96]. A survey of the awareness, understanding, and
implementation of open system concepts within the DoD is available
from the Open Systems Joint Task Force (OSJTF) [OSJTF
96].
There is more experience with the use of COTS items, but often
this experience is with COTS hardware. The concerns, problems, and
solutions for COTS-based software systems are somewhat different and
not as well understood.
An open systems approach requires investments in the following
areas early in a program's life cycle and on an ongoing basis:
- market surveys to determine the availability of standards
- standards selection
- standards profiling- the coordination and tailoring of
standards to work together
- selection of standards-compliant implementations
These costs/activities are the necessary foundation for creating
systems that serve current needs and yet can grow and advance as
technology advances and the marketplace changes.
A separate cost is the continued willingness of the government to
invest in standards development and maturation activities. While
these activities are most often handled at high government levels
concerned with standards development and usage (for example,
Defense Information Systems Agency (DISA) in the
DoD), it is likewise important for individual programs (especially
the larger programs) to stay informed in this area. For example,
individual programs should be concerned about the following
issues:
- When are revisions to specific standards coming out?
- What changes are proposed in the new revision?
- When are ballots on the revisions going to occur?
- Where are the implementations headed?
A COTS-based systems approach also requires new and different
investments:
- market research on available and emerging products and
technologies
- COTS product evaluation and selection
- "black box" integration of COTS components
This technology is classified under the following categories.
Select a category for a list of related topics.
|
[Carney 97a]
|
Carney, D, & Oberndorf, P. "The
Commandments of COTS: Still Searching for the Promised
Land." Crosstalk 10, 5 (May 1997): 25-30. Also
available online at
<URL: http://www.stsc.hill.af.mil/Crosstalk/frames.asp?uri=1997/05/commandments.asp>.
|
|
[Carney 97b]
|
Carney, D. Assembling Large Systems
from COTS Components: Opportunities, Cautions, and
Complexities [online]. Available WWW <URL:
/cbs/papers/paper13a.html>.
|
|
[FAR 96]
|
Federal Acquisition Regulations.
Washington, DC: General Services Administration,
1996.
|
|
[IEWCS 96]
|
Open Systems Joint Task Force Case
Study of U.S. Army Intelligence and Electronic Warfare
Common Sensor (IEWCS) [online]. Available
WWW
<URL:
http://www.acq.osd.mil/osjtf/how_to_do_os/program_apps/index_1b.html>
(1996).
|
|
[Meyers 97]
|
Meyers, Craig & Oberndorf, Tricia.
Open Systems: The Promises and the Pitfalls.
Pittsburgh, PA: Software Engineering Institute, Carnegie
Mellon University, 1997.
|
|
[OSJTF 96]
|
Open Systems Joint Task Force
Baseline Study [online]. Available WWW
<URL:
http://www.acq.osd.mil/osjtf/current_activities/studies_and_projects/baseline.doc>
(1996).
|
Tricia Oberndorf, SEI
- 7 August 97: updated [Carney 97a] reference
- 2 July 97: incorporated new definitions for COTS and NDI from
the FARs
- Minor updates in Maturity section and Costs and
Limitations
- 9 April 97: minor edits; no content changes
- 10 Jan 97 (original); co-author for this version: John
Foreman, SEI
1 "Property" in this definition
explicitly excludes real property.
2 It should be noted that interface
specifications are in general not sufficient to ensure full
"plug-and-play" operation. In practice, the real interface between
two components of a system consists of all the assumptions that each
makes about the other. APIs, data formats, and protocols address a
large number of these assumptions, but by no means all of them. It
remains for further investigations to determine the full set of
interface knowledge that must be standardized to ever get really
close to an ideal "plug-and-play" system creation process.
3 In June 1994 Secretary of Defense
William Perry directed that DoD acquisitions should make maximum use
of performance specifications and commercial standards. In November
1994 Undersecretary of Defense (Acquisition and Technology) Paul
Kaminski directed "that `open systems' specifications and standards
be used for acquisition of weapon systems electronics to the greatest
extent practical."
