Software Engineering Institute Carnegie Mellon

Open Systems
FAQ
Bibliography
Glossary
Promises & Pitfalls

Annotated Bibliography for Open Systems

Most annotations contained herein are taken verbatim from the reference they annotate and/or from accompanying memoranda or similar cover letters.

[AJPO 83]
Ada Joint Program Office. ANSI/MIL-STD-1815A, Reference Manual for the Ada Programming Language. Defense Information Systems Agency, 701 S. Courthouse Rd., Arlington, VA 22204, January 1983.

This standard specifies the form and meaning of program units written in Ada. Its purpose is to promote the portability of Ada programs to a variety of data processing systems. Ada is a programming language designed in accordance with requirements defined by the United States Department of Defense. The language includes facilities offered by classical languages such as Pascal as well as facilities often found only in specialized languages. It is a modern algorithmic language with the usual control structures and with the ability to define types and subprograms. It also serves the need for modularity, whereby data, types, and subprograms can be packaged. It treats modularity in the physical sense as well, with a facility to support separate compilation. It also covers real-time programming, systems programming, and both application-level and machine-level input/output.

[Breuker 94]
Breuker, Dave; Good, Katherine; & Colangelo, Paula. Software Technology to Lower Total Cost of Ownership of C3I Systems. Presentation at the BMD C3 Conference at USAFA, Martin Marietta Corporation, Granite Sentry Program, 2025 Research Parkway, Colorado Springs, CO 80920, March 1994.

In today's world of budget constraints and substantial downsizing, reducing the total cost of ownership while maintaining current operational capabilities and incorporating changing mission requirements is a significant challenge. This paper presents a strategy that can substantially reduce the cost of ownership of C3I systems. An integrated product and process approach, strict adherence to standards, and optimal use of commercial and government off-the-shelf (COTS/GOTS) products provide significant cost reduction by reducing expensive human intensive engineering activities.

[Buddenberg 93]
Buddenberg, Rex. Necessary ... but not Sufficient. Report prepared for Space and Naval Warfare Systems Command (SPAWAR) by the Naval Postgraduate School through the Naval Research and Development Center (NRaD) of the Naval Command, Control, & Ocean Surveillance Command (NCCOSC), San Diego, CA 92152-5000, October 1993.

This review of the Next Generation Computer Resources (NGCR) Program, written for the NGCR Program Office, describes some of the DoD barriers to realizing open systems and provides recommendations for the Navy to grow into an open system architecture for its combat information systems. Recommendations focus on organizational commitment, education of the people involved, and the development of an information system model.

[Clapp 91]
Clapp, Judith A. & Stanten, Saul F. A Guide to Total Software Quality Control (MTR11284), two vols. Griffiss Air Force Base, NY: Mitre Corporation Air Force Materiel Command, 1991.

[Collins 91]
Collins, James F. Letter to Dr. T. Michael Elliott, Executive Director, IEEE Computer Society, June 27, 1991.

This letter distributes the results of a meeting between several large international companies to discuss their shared requirement for interoperable computing environments. They shared a belief that this could best be achieved through open systems-defined as products based on widely implemented vendor-neutral standards. The group agreed upon a common set of open systems requirements and a proposed implementation process that they communicated to vendors and others in the industry. The results included a loose profile of standards and the time frames in which the companies expected to be moving to them.

[CRIB]
Ross, Keey J., DeAngelis, Chris, Teresko, Rheta. The CRIB Expert System for Integrating Open System Components. Naval Undersea Warfare Center, Division Newport, Code 2253, Newport, RI 02841, 1995.

This paper describes the Computer resources Information Base (CRIB), a rule-based expert system/tool designed to address the need for open-systems integration specialists. It assists the user in specifying requirements for each component, selecting a limited number of open standard compliant products that meet those requirements, and integrating multiple interoperable components into a computer system. CRIB maintains a growing database of currently available COTS computer hardware and software products, provides performance and historical information on these products, generates product comparisons, and supports the development of effective configurations of these hardware and software components.

[DFARS]
Defense Federal Acquisition Regulation Supplement (DFARS), Section 210.001

[DoD 85]
U.S. Department of Defense. DoD 4245.7M, Transition from Development to Production ... Solving the Risk Equation. Director, Major Systems Acquisition, Assistant Secretary of Defense for Acquisition and Logistics, Room 2A330, The Pentagon, Washington, DC 20301 (NTIS #AD-A161 063), September 1985.

This manual provides assistance in structuring technically sound programs, assessing their risk, and identifying areas needing corrective action. It is designed to facilitate engineering discipline that will help the DoD collectively make wiser decisions on ongoing programs. It provides a number of "templates." These templates are provided to introduce discipline into the system, to identify and give visibility to high risk factors, and then to provide the tools by which risk can be minimized progressively.

[DoD 87]
1987 Defense Authorization Act.

[DoD 88]
U.S. Department of Defense. DoD-STD-2167A, Defense System Software Development. Commander, Space and Naval Warfare Systems Command, ATTN: SPAWAR-3212, Washington, D.C. 20363-5100, February 29, 1988.

This standard establishes uniform requirements for software development that are applicable throughout the system life cycle. The requirements of this standard provide the basis for government insight into a contractor's software development, testing, and evaluation efforts. This standard is not intended to specify or discourage the use of any particular software development method. This standard, together with the other DoD and military documents referenced within, provides the means for establishing, evaluating, and maintaining quality in software and associated documentation. The applicable data item descriptions (DIDs) are included. They describe a set of documents for recording the information required by this standard. Tailoring guidance can be found in DoD-HDBK-248, Guide for Application and Tailoring of Requirements for Defense Material Acquisitions.

[DoD 91a]
U.S. Department of Defense. DoDD-5000.1, Defense Acquisition. Washington, DC: Deputy Secretary of Defense, February 23, 1991.

This directive establishes a disciplined management approach for acquiring systems and materiel that satisfy the operational user's needs. The policies it promulgates establish a disciplined approach for integrating the efforts and products of the Department's requirements generation; acquisition management; and planning, programming, and budgeting systems. It cancelled 80 previous standards and instructions, providing consolidation and simplification, replacing them with DODI 5000.2.

[DoD 91b]
U.S. Department of Defense. DODI-5000.2, Defense Acquisition Management Policies and Procedures. Washington, DC: Undersecretary of Defense for Acquisition, February 23, 1991.

DoD acquisition management policies and procedures have traditionally been published in numerous separate directives and instructions. These documents were typically supplemented by the DoD components. Over time this resulted in a heavily cross-referenced maze of guidance that stifled creativity and individual judgement and defied practical use. This instruction seeks to remedy that problem by establishing a core of fundamental policies and procedures that can be implemented down to the program manager and field operating command level without supplementation. The information in this instruction was condensed from over 45 separate DoD issuances that have been canceled and countless DoD component publications that are being canceled.

[DoD 91c]
U.S. Department of Defense. DODIIS (Department of Defense Intelligence Information System) Reference Model for the 1990s (draft). May 14, 1991.

The DODIIS reference model is intended for use by DODIIS planners and engineers responsible for preparing or directing DODIIS upgrades. Despite the title, it actually contains more than just the reference model; it includes a selection of several standards, which it categorizes according to the reference model. These standards are recommended for use. It is based largely on the NIST APP (National Institute of Standards and Technology Application Portability Profile) [NIST 93b].

[DoD 93]
U.S. Department of Defense. DoD 5000.37M, Commercial and Other Non-Developmental Items (NDI) Manual (draft). Director, Manufacturing Modernization, Room 3B253, The Pentagon, Washington, DC 20301-8000, September 1993.

This manual has been written as a guide for acquisition managers and functional personnel who are, or will be, involved in NDI acquisitions. Users of this manual should be familiar with the acquisition process and have an understanding of government contracting. The manual contains information intended to help implement NDI acquisitions, without inhibiting creative and innovative strategies. Where checklists are provided, they are intended for guidance only. In using these checklists, tailor and supplement them to meet the unique circumstances of each acquisition.

This manual provides guidance for increasing the use of NDI across the entire spectrum of acquisitions. This manual does not stipulate step-by-step procedures to follow in acquiring NDI. Rather it describes tools and techniques for increasing the use of NDI and avoiding common pitfalls. This approach permits the manual to be used on a broad range of acquisitions and allows DoD agencies to issue service-specific guidance as necessary.

[DoD 94a]
U.S. Department of Defense. Technical Architecture Framework for Information Management (TAFIM), Volumes 1-8, Version 2.0. DISA Center for Architecture, 10701 Parkridge Blvd., Reston, VA 22091-4398, June 30, 1994.

The TAFIM is an eight-volume set of documents produced by DISA to define the structure within which it manages the use of standards in the DoD, particularly within the automated information system and the command, control, and intelligence arenas. Volume 2 contains a technical reference model and standards profile summary; other volumes provide information that complements this volume, such as a standards-based architecture planning guide (Volume 4), a goal security architecture (Volume 6), and a human-computer interface style guide (Volume 8). Volume 7 focuses on Adopted Information Technology Standards (AITS) and is shown separately in this bibliography under [DoD 94b].

[DoD 94b]
U.S. Department of Defense. Technical Architecture Framework for Information Management, Volume 7, Adopted Information Technology Standards (AITS), Version 2.0. DISA Center for Architecture, 10701 Parkridge Blvd., Reston, VA 22091-4398, August 2, 1994.

The AITS provides a consolidated listing of DoD adopted information technology standards. It is a product of the DoD Defense Information Systems Agency (DISA). The objective of the AITS is to provide consistent standards guidance across the Enterprise, Mission, Function, and Application levels of the DoD Integration Model (a reference model described in Volume 1 of the TAFIM). It provides the current set of DoD adopted standards mature enough to merit broad-scale use across DoD programs. Emerging, or goal, standards that are predictable replacements for adopted standards are identified via footnote or annotation to guide planning decisions as far into the future as possible.

[DoD 95a]
U.S. Department of Defense. DoDD-5000.1, Defense Acquisition. Washington, DC: Deputy Secretary of Defense, February 23, 1991.

This document updates [DOD 91a].

[DoD 95b]
U.S. Department of Defense. Draft DODI-5000.2, Mandatory Procedures for Major Defense Acquisition Programs (MDAPs) and Major Automated Information System Acquisition Programs (MAISAPs). Washington, DC: Under Secretary of Defense (Acquisition & Technology), October 18, 1995.

This document updates [DOD 91b]. It incorporates new laws and policies, including FASA and the institutionalization of Integrated Product Teams (IPTs), separates mandatory policies and procedures from discretionary practices, responds to the perception that the acquisition policy documents have grown unwieldy and too complex, and integrates acquisition policies and procedures for both weapon systems and automated information systems (AISs). Major themes of this instruction are teamwork, tailoring, empowerment, cost as an independent variable (CAIV), use of commercial products, and best practices.

[DoN 93]
U.S. Department of the Navy. Nondevelopmental Item (NDI) Acquisition Improvement Working Group, Revised Draft Final Report. Washington, DC: July 29, 1993. Although this working group started out to shorten the time required to acquire and deploy NDI by revising established DoD policy to allow the use of NDI, they came to realize that the policy is not what’s “broken.” The focus of the report is on documenting this conclusion and propagating this knowledge to the acquisition workforce. The goal is to change the perception that alternative acquisition approach(es) are higher risk, to one in which program managers realize that they can be substantially rewarded for taking full advantage of alternatives authorized in DoD 5000 series to abridge the acquisition cycle and shorten program schedule.

[DODISS 95]
Index of Specifications and Standards. Part 1. Alphabetical Listing. Assistant Secretary of Defense (Command, Control, Communications, and Intelligence), Office of the Assistant Secretary of Defense for Economic Security. Washington, DC: July 1995.

The DODISS is a reference publication available in a variety of formats. It is composed of four parts and contains catalog listings of military specifications and standards, federal specifications and standards, military handbooks, qualified products lists (QPLs), commercial item descriptions (CIDs), DoD adopted non-government/industry documents, Air Force/Navy aeronautical standards, Air Force-Navy aeronautical design standards, Air Force specifications bulletins, and cancellation lists and other departmental documents.

[DSB 87]
Office of the Undersecretary of Defense for Acquisition. Final Report of the Defense Science Board, 1986 Summer Study. January 1987.

[DTIC 93]
Defense Technical Information Center. Streamlining Defense Acquisition Laws. Report of the Acquisition Law Advisory Panel to the United States Congress (AD-A262 699). Alexandria, VA: January 1993.

This report provides the details of the work of the DoD Advisory Panel on Streamlining and Codifying Acquisition Law; it is summarized in [FCR 93]. The report is over 1800 pages long, 3.5 inches thick, and weighs about 12 pounds. In spite of that, it contains a lot of interesting reading.

[FARs] - Federal Acquisition Regulations

[FASA 1994]
Federal Acquisition Streamlining Act of 1994

This legislation was passed by Congress in 1994 to revise and streamline the acquisition laws of the Federal Government, and for other purposes. It amends several previously pieces of legislation and regulations.

[FCR 93]
DoD Advisory Panel on Streamlining and Codifying Acquisition Laws. For a reference to the results of the group, see Federal Contracts Reports, Feb. 1, 1993, pages 125-139.

Also known as the Section 800 panel, this group was put together for the purpose of making recommendations on streamlining and improving government acquisition. Among their recommendations:

The FASA and Perry memo were based to a large extent on the recommendations from this panel. The details of the report are in [DTIC 93].

[Ferguson 94]
Ferguson, Jack & DeRiso, Michael E. Software Acquisition: A Comparison of DoD and Commercial Practices (CMU/SEI-94-SR-9). Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University, October 1994. Draft.

This report compares best commercial practice with the current DoD acquisition process for acquiring software and recommends some steps that can be taken to streamline DoD software acquisitions to minimize overall life-cycle costs.

[Fisher 94]
Fisher, Gary E. Guide on Open System Environment (OSE) Procurements. Systems and Software Technology Division, Computer Systems Laboratory, NIST, Gaithersburg, MD 20899, May 1994.

This report provides agency program managers, systems engineers, and contracting officers with a model for developing the plans and specifications necessary to define the open system environment (OSE) requirements in requests for proposals (RFPs), based on real open system procurements that have been used over the last three years. Additional information is provided for assisting agencies in determining which portions of the report are applicable to their acquisition plans. Lessons learned are highlighted in annotated text that accompanies many of the report’s subsections. The guidance in the report pertains to U.S. Government acquisition of OSE software infrastructure including operating system, human/computer interface, software engineering, data management, data interchange, graphics, network, security, and system/network management services. The guidance is based on implementations of standard application program interfaces, programming languages, data formats, and protocols. Other organizations, such as state and local governments and academic and private institutions, may also find the information helpful in defining computing environments that promote application portability, interoperability, and scalability.

[GOSIP 91]
Government Open Systems Interconnection Profile (GOSIP), FIPS 146-1, Version 1. National Technical Information Service, U.S. Department of Commerce, Springfield, VA 22161. April 1991.

FIPS 146-1 adopted the Government Open Systems Interconnection Profile (GOSIP), which defines a common set of Open Systems Interconnection (OSI) protocols that enable systems developed by different vendors to interoperate and the users of different applications on those systems to exchange information.

The primary objectives of this standard are to

The Industry Government Open Systems Specification (IGOSS) issued as NIST Special Publication 500-217 updates the OSI protocols in FIPS 146-1 and may be used by federal government agencies when they wish to acquire computer networking products and services and communications systems or services that are based on OSI standards.

NOTE: Changes to FIPS 146-1, GOSIP, have been proposed, with a commenting period that closed on October 27, 1994. The proposed changes rename the FIPS as Profiles for Open Systems Internetworking Technologies and modify the standard by removing the requirement that federal agencies specify the GOSIP protocols when agencies acquire networking products and services and communications systems and services. The revision, which will be issued as FIPS 146-2, provides references to additional specifications that agencies may use in the acquisition of open systems, such as those issued by the IETF, the ITU, and ISO, may be used.

[Howell 90a]
Howell, Steve. Evaluation Process Report for Next Generation Computer Resources Operating Systems Interface Baseline Selection (NAVSWC TR 90-248). NGCR, OSSWG, Naval Surface Warfare Center, Silver Spring, MD 20903-5000, May 7, 1990.

This report documents the techniques and methods by which the NGCR Operating Systems Standards Working Group (OSSWG) derived a recommended baseline operating system interface specification. It describes the process, the inputs into the process in the form of criteria, baseline candidates, evaluators, and weights, as well as the outputs of the process. A taxonomy that describes the relationships and organizations of the criteria is presented as well as the procedure by which the candidate baselines were to be evaluated.

[Howell 90b]
Howell, Steve. Evaluation Results Report for Next Generation Computer Resources Operating Systems Interface Baseline Selection (NAVSWC TR 90-246). NGCR, OSSWG, Naval Surface Warfare Center, Silver Spring, MD 20903-5000, May 7, 1990.

This report summarizes the results of the NGCR OSSWG evaluation of candidates for the Operating System Interface (OSIF) Baseline.

[IEEE 91]
Institute of Electrical and Electronic Engineers (IEEE). Standard Glossary of Software Engineering Terminology, IEEE Std 610.12-1990. IEEE Computer Society Standards Coordinating Committee, IEEE, 345 East 47th Street, New York, NY 10017, with corrections dated January 22, 1991.

This standard identifies terms currently in use in the field of software engineering. Standard definitions for these terms are established.

[IEEE 93a]
Institute of Electrical and Electronic Engineers (IEEE). IEEE Standard for Futurebus+, Profile M (Military), IEEE Std 896.5-1993. IEEE, 345 East 47th Street, New York, NY 10017, June 17, 1993.

Futurebus+ standards provide systems developers with a set of tools with which high performance bus-based systems may be developed. This architecture provides a wide range of performance scalability over both cost and time for multiple generations of single- and multiple-bus multiprocessor systems. This document, a companion standard to IEEE Std 896.1-1991, builds on the logical layer by adding requirements for three military profiles. It is to these profiles that products will claim conformance.

[IEEE 93b]
Institute of Electrical and Electronic Engineers (IEEE). Draft Guide for Information Technology - Portable Operating System Interface (POSIX) - The Open Systems Environment, P1003.0/D16 (also known as POSIX.0). Portable Applications Standards Committee of the IEEE Computer Society, IEEE, 345 East 47th Street, New York, NY, 10017, August 1993.

This guide presents an overview of open system concepts and their application. Information is provided to persons evaluating systems on the existence of, and interrelationships among, application software standards, with the objective of enabling application portability and system interoperability. A framework is presented that identifies key information system interfaces involved in application portability and system interoperability and describes the services offered across these interfaces. (This part is known as the POSIX.0 OSE reference model.) Standards or standards activities associated with the services are identified where they exist or are in progress. Gaps are identified where POSIX open system environment (OSE) services are not currently being addressed. Finally, the OSE profile concept is discussed with examples from several application domains.

[IEEE 93c]
Institute of Electrical and Electronic Engineers (IEEE). IEEE Draft Standard P1003.18/D10, POSIX Platform for USSI-P001. IEEE, 345 East 47th Street, New York, NY 10017, 1993.

[ISO 84]
International Organization for Standardization. ISO/IEC 7498: Information Processing Systems - Open Systems Interconnection - Basic Reference Model. 1984.

[ISO 91]
International Organization for Standardization. ISO/IEC 9126: Information Technology — Software Product Evaluation — Quality Characteristics and Guidelines for Their Use. 1991.

[JIEO 94]
Joint Interoperability and Engineering Organization. Information Technology Standards Guidance (ITSG), Version 2.0 (11 parts). JIEO circular. 9/30/94.

The ITSG contains implementation guidance that is intended to accompany the AITS [DoD 94b]. It provides additional, supporting information about the AITS standards and also identifies additional standards not contained in the AITS because of deficiencies, lack of maturity, or other factors that preclude their inclusion in the AITS. It also provides recommendations for specifying standards in system acquisition documents.

[Kowalski 94]
Kowalski, Norm & Chestnutwood, Mark. Transition To Open Standards. Tutorial for International Test and Evaluation Association (ITEA) 1994, Second Annual Test Technology Transfer Symposium, August 29, 1994.

This tutorial covers key open system concepts and definitions, an open system engineering process, open system policy, conformance management, today’s transition environment, and an open system transition process. Both authors work for the Navy and have been involved in the NGCR program as well as the application of these concepts to real systems.

[Kuhn 70]
Kuhn, Thomas, The Structure of Scientific Revolutions. Chicago: University of Chicago Press, 1970.

[Mellor 92]
Mellor, Peter. Critique of ISO/IEC 9126 (ISO/IEC JTC1/SC7 N1028). London: City University, Centre for Software Reliability, 1992.

[Marcantonio 94]
Marcantonio, Al. OSEM - Open Standard Electronic Modules. Raytheon Equipment Division, February 1994.

This paper and brief describe the open system architecture and the dollar and technology benefits derived from commercial standards leveraging commonality between systems or components, emphasizing particularly these arguments for hardware components. The brief includes recommendations for both industry and government.

[McKamey 93]
McKamey, Jerry. Lessons Learned - Navigation and Fire Control; Trident II Commercial Architectures, (briefing), August 19, 1993.

This set of briefing charts describes the lessons learned on the Trident II system in the application of open systems principles. It explores various vendor, implementation, and software issues encountered on this program.

[MIL-HDBK-59B]
Department of Defense Computer-Aided Acquisition and Life-Cycle Support (CALS) Program Implementation Guide

[MIL-STD-490A]
MIL-STD-490A, Specification Practices. Headquarters, Air Force Systems Command, HQ AFSC/SDXP, Andrews Air Force Base, Washington, DC 20331, June 4, 1985.

This military standard sets forth practices for the preparation, interpretation, change, and revision of program-peculiar specifications prepared by or for the departments and agencies of the Department of Defense. It was prepared to establish uniform specification practices in response to the need for a document comparable to DoD-STD-1000 covering engineering drawing practices and in recognition of the configuration identification concepts of the DoD Configuration Management Program.

[MIL-STD-499A]
MIL-STD-499A, (USAF), Engineering Management. Commander, Air Force Systems Command, ATTN: SDDE, Andrews AFB, Washington, DC 20331, May 1, 1974.

This standard was developed to assist government and contractor personnel in defining the systems engineering effort in support of defense acquisition programs. The fundamental concept of this standard is to present a single set of criteria against which all may propose their individual internal procedures as a means of satisfying engineering requirements. It provides program managers
(a) Criteria for evaluating engineering planning and output.
(b) A means for establishing an engineering effort and a systems engineering management plan (SEMP).
(c) Task statements that can be selectively applied to an acquisition program.

[MIL-STD-970]
MIL-STD-970, Standards and Specifications, Order of Preference for the Selection of. Director, Defense Standardization Program Office (DSPO), 5203 Leesburg Pike, Suite 1403, Falls Church, VA 22041-3466, October 1, 1987.

This standard sets forth the order of preference for the selection of standards and specifications used in the design, acquisition, construction, and support of materiel for the Department of Defense in compliance with the Office of Management and Budget (OMB) Circular A-119 on using commercial practices and non-government standards.

[MIL-STD-1388-1A]
MIL-STD-1388-1A, Logistic Support Analysis. April 11, 1983. (Note that revision B of this standard is now available.)

This standard implements the logistic support analysis (LSA) guidelines and requirements established by Department of Defense (DoD) Instruction 5000.2, Major System Acquisition Procedures. The requirements of this standard are applicable to major and less-than-major system/equipment acquisition programs, major modification programs, and applicable research and development projects. The goal of this standard is a single, uniform approach by the military services for conducting those activities necessary to (a) cause supportability requirements to be an integral part of system requirements and design, (b) define support requirements that are optimally related to the design and to each other, (c) define the required support during the operational phase, and (d) prepare attendant data products. This standard provides general requirements and descriptions of tasks which, when performed in a logical and iterative nature, compose the LSA process. The tasks are structured for maximum flexibility in their application. In addition to the general requirements and task description sections, this standard contains an application guidance appendix that provides rationale for selecting and tailoring the tasks to meet program objectives economically.

[MIL-STD-2036A]
MIL-STD-2036A, General Requirements for Electronic Equipment Specifications. September 3, 1993.

This standard covers the policy guidance, and general and detailed requirements, for the preparation of specifications for electronic equipment used in shipboard (including submarines) and space applications. This document provides guidance for the use of commercial-off-the-shelf (COTS), ruggedized, and militarized equipment. Requirements shall be based on the Installation and intended use of the equipment.

[NATO 93]
North Atlantic Treaty Organization (NATO) Ada Implementation Sub Group (AISG) of the ISWG. NATO Open Systems Environment Reference Model. NATO ISWG/AISG OSE Expert Group, June 23, 1993.

NATO open systems standard interfaces, protocols, services, and supporting formats will have to be defined within an overall reference model context. This reference model is necessary to establish a context for understanding how the disparate technologies required as part of a future NATO open systems environment relate to each other and to provide a mechanism for identifying the key issues associated with application software portability and interoperability. The NATO OSE reference model does not impose any architectural constraints. Its purpose is to provide a common conceptual framework, define a common vocabulary, and specify a base of standards for NATO project and procurement staff. The NATO OSE reference model adopts the POSIX OSE reference model (in the POSIX.0 Guide) in its basic form.

[NAWC 93]
Naval Air Warfare Center, Aircraft Division, Warminster. Benefits and Risks of COTS Requirements for Automated System Acquisitions. Systems and Software Technology Department, Naval Air Warfare Center, Aircraft Division, Warminster, PA 18974-5000, April 15, 1993.

It is the goal of this paper to present a balanced perspective of the benefits and risks associated with the use of COTS components in the acquisition of a system and at the same time encourage program managers to begin assessing the risks at the earliest stages of systems design. To successfully use COTS components in the development of a system, it is necessary to implement a risk management program with the strategy and resources to deal with potential problems that may result from their use. The cost of using COTS components is more than simply the up-front purchase price. There are ongoing costs associated with the use of COTS components that must be factored into the projected system development and maintenance budgets.

[NCCS 88]
Naval Command and Control System Afloat. Standardization Guidelines for Developing NCCS Afloat Subsystems, Preliminary, November 23, 1988.

This document provides design standardization guidelines consistent with open systems design for use in the development and implementation of Naval Command and Control System (NCCS) Afloat subsystems. Application of these guidelines is intended to influence the NCCS Afloat design to be flexible, allowing economical enhancements during its life cycle to respond to changes in operational requirements and new technology. It establishes an open system design approach that permits system definition through the use of government and public domain industry standards for hardware and software. It describes the choices made for software standards (e.g., operating systems, programming language), software documentation requirements, and hardware standards (e.g., backplane bus, peripheral interface).

[NDI 87]
NDI Preference Act. 1987.

[NGCR 93a]
Next Generation Computer Resources (NGCR) Program. Life Cycle Cost/Benefit Impact Assessment. NGCR Program, SPAWAR 331, 2451 Crystal Park 5, Suite 701, Washington, D.C. 20363-5200 (new address: Arlington, VA 22245-5200), May 21, 1993.

The objective of this analysis is to identify the overall cost and benefit ramifications of the NGCR standards to the U.S. Navy in aggregate. This cost/benefit analysis is designed to provide the NGCR program manager and other decision-makers with a better understanding of the magnitude and distributional effects of these perceived costs and benefits. This report has been prepared to answer life-cycle cost/benefit questions, address economic issues, and confirm or reject the intuitive decision that implementation of the NGCR Program will result in overall net benefits to the Navy.

[NGCR 93b]
Next Generation Computer Resources (NGCR) Program. Report on the Progress of the NGCR PSESWG. NGCR Program, SPAWAR 331, 2451 Crystal Park 5, Suite 701, Washington, D.C. 20363-5200 (new address: Arlington, VA 22245-5200), September 1993.

This progress report was written by the NGCR Project Support Environment Standards Working Group (PSESWG) to capture the state of its accomplishments and what they might mean for two potential kinds of readers: those who might come along in the future to continue the original PSESWG charter, and those who are trying today to select interface standards for PSE projects. It focuses on the work done since the development of the PSE reference model, the PSE available technology report, and the PSESWG initial selection of a few interface standards. This work includes several mappings to the PSE reference model, some investigations into framework standards and data interface issues, and recommendations for transitioning to open system PSEs.

[NGCR 93c]
Next Generation Computer Resources (NGCR) Program. User’s Guide. NGCR Program, SPAWAR 3312, 2451 Crystal Park 5, Suite 701, Washington, D.C. 20363-5200 (new address: Arlington, VA 22245-5200), September 30, 1993.

This guide addresses the acquisition and standardization issues relevant to the implementation of the NGCR standards for any given application. Service classes covered by this guide include open computer backplane bus architectures, networks, and the POSIX operating system interface standards. For each service class, the guide discusses engineering guidance, acquisition issues, application issues, implementation issues, and standardization issues.

[NGCR 93d]
Next Generation Computer Resources (NGCR) Program. User’s Guide, Draft Supportability Volume. NGCR Program, SPAWAR 331, 2451 Crystal Park 5, Suite 701, Washington, D.C. 20363-5200 (new address: Arlington, VA 22245-5200), September 30, 1993.

The focus of this supportability volume of the User’s Guide is to provide the reader with helpful, cost-effective information that will capitalize on the inherent benefits of using NGCR interface standards in a military environment. It is an advisory document for use by program management offices and their supporting warfare centers and contractors. It includes general guidance relative to supportability functions at the end-item product or system level and specific guidance related to support of Futurebus+, SAFENET, and POSIX-based products. Where possible and appropriate, this volume provides lessons learned in application of NGCR-specific or similarly supportable products.

[NGCR 95a]
Next Generation Computer Resources (NGCR) Program. Draft Open Systems Computer Resources Supportability Guide. NGCR Document No. AST 001 ver 0.12. NGCR Program, Space and Naval Warfare Systems Command, SPAWAR 331, 2451 Crystal Drive, Arlington, VA 22245-5200, 17 April 1995.

[NGCR 95b]
Next Generation Computer Resources (NGCR) Program. Draft Open Systems Computer Resources Supportability Guide. NGCR Document No. AST 002 ver 0.10. NGCR Program, Space and Naval Warfare Systems Command, SPAWAR 331, 2451 Crystal Drive, Arlington, VA 22245-5200, 15 September 1995.

The Open Systems Computer Resources Supportability Guide provides general guidance on supportability analysis and planning for open system interface standards based products in tactical systems. The guidance presented is: an introduction to open systems; integrated logistic support elements and interfaces in an open systems environment; and specific support information on selected open system interface standards.

[NIST 93a]
National Institute of Standards and Technology. FIPS 151-2, Portable Operating System Interface (POSIX) - System Application Program Interface [C Language]. Gaithersburg, MD: NIST, Computer Systems Laboratory, Gaithersburg, MD 20899, May 12, 1993.

This standard defines a C programming language source interface to an operating system environment. This standard is for use by computing professionals involved in system and application software development and implementations. This revision supersedes FIPS PUB 151-1 in its entirety.

The FIPS PUB 151-2 specifications are the specifications contained in the international standard ISO/IEC 9945-1:1990, Information Technology - Portable Operating System Interface (POSIX) - Part 1: System Application Program Interface (API) [C Language], with the modifications specified in section 12 of FIPS 151-2. These modifications ensure that applications, that choose to use those optional features specified in POSIX.1 and mandated below, are strictly conforming FIPS 151-2 applications (portable to all conforming FIPS 151-2 implementations). For each modification a reference to the associated POSIX text is provided. There are 16 modifications given.

[NIST 93b]
National Institute of Standards and Technology, Application Portability Profile (APP): The U.S. Government’s Open System Environment Profile - OSE/1 Version 2.0, NIST Special Publication 500-210. Systems and Software Technology Division, Computer Systems Laboratory, NIST, Gaithersburg, MD 20899, June 1993.

This document is directed toward helping users make an informed judgement regarding the choice of specifications to meet current requirements, particularly in those areas where formal standards do not exist. There are two dimensions of the assistance provided. First, specifications are provided for each functional area of the APP. The specifications represent the collective judgement of the NIST staff regarding the most appropriate specification for each functional area. Second, and equally as important, evaluation criteria to assist in making a qualitative assessment of the recommended specifications are defined and applied. Application of these evaluation criteria provides the NIST assessment of the suitability of the specifications recommended. Users should use the evaluation criteria to make their own assessments of the recommended specifications.

[Nutt 92]
Nutt, Gary. Open Systems. Englewood Cliffs, NJ: Prentice Hall, 1992.

This book examines the evolving corporate computer environments to understand why open systems are currently critical to the corporation. Then it goes on to describe architectural approaches to open systems and some of the standards involved.

[NUWC 95]
Naval Undersea Warfare Center. New Attack Submarine Open System Implementation, Specification and Guidance. NUWC-NPT Technical Document 10,414A, NUWC (Norm Kowalski, compiler), Code 222, Bldg 1171, Newport, RI 02841-5049. 29 August 1995.

This document provides the New Attack Submarine (NSSN) Open Systems Architecture (OSA) Policy, Standards, Profiles, Implementation Conformance Statements, and Guidance for the utilization of open standards in the NSSN. It includes detailed profiles for POSIX.1, networks, backplanes, graphics and windowing, databases, and system management. It also discusses conformance considerations for these profiles and provides general guidance.

[Ortiz 92]
Ortiz, Alberto. System Engineering Concept Demonstration, Process Model (RL-TR-92-345), Vol III (of seven). McDonnell Douglas Corporation for Rome Laboratory, Air Force Materiel Command, Griffiss Air Force Base, NY, 13441-4505, 1992.

This volume presents the details and results of the development of a system acquisition and development model that emphasizes systems engineering-oriented tasks and activities over the entire system life cycle. The purpose of the model was to 1) promote better understanding of the tasks and activities of the systems engineering process as imposed by various MIL/DoD/AF standards and regulations and 2) support the determination of completeness and adequacy of the envisioned systems engineering automation’s system-level requirements.

[OTA 92]
U.S. Congress, Office of Technology Assessment. Global Standards: Building Blocks for the Future (TCT-512). Washington, DC: U.S. Government Printing Office, March 1992.

This study considers the U.S. standards-setting process in light of the changing economic and technological environment. Looking across industry sectors, the study compares the U.S. system with those of other countries, particularly the European Economic Community (EEC). Where remedies seem to be warranted, the study suggests alternative strategies and options that the United States might pursue.

[Packard 86]
President’s Blue Ribbon Commission on Defense Management (Packard Commission). A Report to the President on Defense Acquisition. April 1986.

[Roark 94]
Roark, Chuck & Kiczuk, Bill. Transitioning Open Systems Standards to the DoD. Texas Instruments Defense Systems & Electronics Group, March 1994.

There is a growing desire within the DoD to use open systems standards. Several groups within the DoD ... and industry sponsored standards committees ... are exploring the transition of open systems standards to DoD use, with particular emphasis for mission-critical applications. This paper explores the pros and cons for use of open systems standards within the DoD and discusses issues in making the transition to open systems standards.

[SD-2 96]
SD-2 Office of the Under Secretary of Defense for Acquisition and Technology. Buying Commercial & Nondevelopmental Items: A Handbook [online]. Available WWW (1996).

[SECNAVINST 5200.32A]
SECNAVINST 5200.32A, Acquisition Management Policies and Procedures for Computer Resources. ASN (RD&A), Office of the Secretary of the Navy, Washington, DC 20350-1000, May 3, 1993.

The purpose of this SECNAVINST is to provide policy for acquiring Department of the Navy computer resources and to establish the internal management processes to comply with DoDD 5000.1 and accompanying policy documents. It states Navy policy for computer resources to be designed, procured, and supported based on an open systems approach and no longer requires programs to use products developed under the standard embedded computer resources (SECR) program (e.g., AYK-14, UYK-43). It includes a Computer Resources Management Cross Reference Guide that relates “key management areas” (e.g., acquisition strategy, acquisition streamlining, configuration management), milestones, and various plans to specific sections of DODI 5000.2 and SECNAVINST 5000.2A.

[SECNAVNOTE 5200]
SECNAVNOTE 5200, ASN (RD&A). Office of the Secretary of the Navy, Washington, DC 20350-1000, May 20, 1993.

This SECNAVNOTE publishes the current Open System Interface Standards List (OSISL) and Products Accepted List (PAL). The OSISL areas include

The PAL includes: AN/AYK-14 (Advanced), AN/UYK-43 [Open Systems Module (OSM)], AN/UYK-44 (OSM).

[Stovall 93]
Stovall, John R. and Richard B. Wray, Implementing the Space Shuttle Data Processing Systems with the Space Generic Open Avionics Architecture. Lockheed Engineering & Sciences Company, Houston, TX, under contract for NASA Johnson Space Center, December 1993.

This paper presents an overview of the application of the Space Generic Open Avionics Architecture (SGOAA) to the Space Shuttle Data Processing Systems architecture design. Section 2 provides an overview of the requirements, design and operation of the Space Shuttle avionics. Section 3 provides an overview of the tailoring of the SGOAA to the needs of the shuttle avionics. Section 4 provides some conclusions.

[TSWG 93]
Tri-Service Working Group. Tri-Service Open Systems Architecture Working Group Final Report Document. Deputy Assistant Secretary of the Navy (Command, Control, Communications, and Computers/Electronic Warfare/Space), Office of the Assistant Secretary of the Navy (Research, Development, and Acquisition), Washington, DC 20350-1000, September 1993.

The purpose of the working group was to: define open systems architecture, its attributes, and criteria for establishing open systems standards; identify a process to develop, adopt, or minimally extend open systems standards; draft a DoD policy directive pertaining to the use of open systems standards in DoD warfare and warfare support systems (WWSS), and develop a framework for use by DoD components in preparing their transition plans to implement the DoD open systems policy. The report covers these results, including a technical reference model for WWSS.

[Wray 93]
Wray, R. B. and Stovall, J. R., Space Generic Open Avionics Architecture (SGOAA) Reference Model Technical Guide. Lockheed, under NASA contract NAS9-17900 for the Johnson Space Center. April 1993.

[XPG4 95]
The X/Open Portability Guide. X/Open Publication Set T906. ISBN 1-85912-083-0. Falls Church, VA: X/Open Company Limited, March 1995.

XPG4 is the name for the collection of volumes that make up the Common Applications Environment (CAE) of X/Open. CAE is a comprehensive open systems framework, and is designed as a vehicle for implementing open systems in the real world. Once X/Open working groups reach consensus, they are adopted into the CAE. A multivolume reference work, the CAE is an evolving portfolio of application programming interfaces, protocols, and specifications. The CAE is also supported with an extensive set of stringent conformance tests and a distinct trademark carried only on those products that comply with X/Open Portability definitions.