Software Engineering Institute

The characterization of software performance (SWP) in complex, service-oriented architecture (SOA)-based system of systems (SoS) environments is an emergent study area. This report focuses on both qualitative and quantitative ways of determining the current state of SWP in terms of both test coverage (what has been tested) and confidence (degree of testing) for SOA-based SoS environments. Practical tools and methodologies are offered to aid technical and programmatic managers:

• a stepwise methodology toward SWP selection
• SWP and system architecture design considerations
• resource limiters of SWP
• SWP and test event design considerations
• organizational and process suggestions toward improved SWP management
• a matrix of measures including test fidelity and realism levels

These tools are not complete, but do offer a good starting point with the intent to encourage contributions to this growing body of knowledge. This report is intended to benefit leaders within the varied acquisition communities, Program Executive Offices, and Program Management Offices. It provides detailed guidance for use by technical leadership as well.