System-of-Systems Test and Evaluation
The Challenge for Test and Evaluation (T&E)
The vision for network-centric warfare (NCW) and network-centric operations (NCO) radically changes the current approach to development and deployment of systems; it also changes the use of systems. "Fitness for use" of individual constituents can only be determined in the context within which a system of systems will be used.
Ultimately, in the service-oriented architecture (SOA) concept, services -- not systems -- will be deployed and composed dynamically into systems. These services become the components of our future systems, but at the time of a service's deployment no one will know or be able to anticipate all the other services and components with which it might be combined.
How, then, can we develop sufficient confidence in the quality of systems of systems? Traditional engineering techniques for both systems and software -- including conventional test and evaluation approaches -- will not be able to completely meet the system-of-systems test and evaluation need for at least the following reasons:
- It is unclear what it means for a system of systems to fail.
- Even given operational scenarios defining the expected capabilities of a system of systems, the amount of testing required to develop any degree of confidence in the system (using the usual statistical approaches) would be prohibitive in time and expense.
- Systems of systems are so large that they are always undergoing changes. It is unclear what techniques should be used to ensure that changes do not have unexpected emergent effects.
- Different users expect/need different qualities of service in different situations.
The Dynamic Systems Program is investigating ways to advance the technology for developing sound conclusions about the functionality, dependability, performance, and safety of systems of systems. For more information about the work, contact Don Beynon: dbeynon@sei.cmu.edu or 412-268-1711 (in the United States).
For more information about the Dynamic Systems Program or the SEI, contact
SEI Customer Relations
Phone: 412-268-5800
FAX: 412-268-5750
Email: customer-relations@sei.cmu.edu