Software Engineering Institute Carnegie Mellon

Ultra-Large-Scale Systems

Executive Summary

The following executive summary is excerpted from Ultra-Large-Scale Systems: The Software Challenge of the Future. See Get the Report for a PDF file of the full report. If you would like more information about ULS systems and the ULS Systems Study, please contact Linda Northrop at lmn@sei.cmu.edu.

What are ULS systems?

The U. S. Department of Defense (DoD) has a goal of information dominance—to achieve and exploit superior collection, fusion, analysis, and use of information to meet mission objectives. This goal depends on increasingly complex systems characterized by thousands of platforms, sensors, decision nodes, weapons, and warfighters connected through heterogeneous wired and wireless networks. These systems will push far beyond the size of today’s systems and systems of systems by every measure: number of lines of code; number of people employing the system for different purposes; amount of data stored, accessed, manipulated, and refined; number of connections and interdependencies among software components; and number of hardware elements. They will be ultra-large-scale (ULS) systems.

How are ULS systems different?

The sheer scale of ULS systems will change everything. ULS systems will necessarily be decentralized in a variety of ways, developed and used by a wide variety of stakeholders with conflicting needs, evolving continuously, and constructed from heterogeneous parts. People will not just be users of a ULS system; they will be elements of the system. Software and hardware failures will be the norm rather than the exception. The acquisition of a ULS system will be simultaneous with its operation and will require new methods for control. These characteristics are beginning to emerge in today’s DoD systems of systems; in ULS systems they will dominate. Consequently, ULS systems will place unprecedented demands on software acquisition, production, deployment, management, documentation, usage, and evolution practices.

Challenges of ULS systems

Fundamental gaps in our current understanding of software and software development at the scale of ULS systems present profound impediments to the technically and economically effective achievement of the DoD goal of deterrence and dominance based on information superiority. These gaps are strategic, not tactical. They are unlikely to be addressed adequately by incremental research within established categories. Rather, we require a broad new conception of both the nature of such systems and new ideas for how to develop them. We will need to look at them differently, not just as systems or systems of systems, but as socio-technical ecosystems. We will face fundamental challenges in the design and evolution, orchestration and control, and monitoring and assessment of ULS systems. These challenges require breakthrough research.

The SEI's ULS research agenda

We propose a ULS systems research agenda for an interdisciplinary portfolio of research in at least the following areas:

The proposed research does not supplant current, important software research but rather significantly expands its horizons. Moreover, because we are focused on systems of the future, we have purposely avoided couching our descriptions in terms of today’s technology. The envisioned outcome of the proposed research is a spectrum of technologies and methods for developing these systems of the future, with national-security, economic, and societal benefits that extend far beyond ULS systems themselves.

Though our research agenda does not prescribe a single, definitive roadmap, we offer three structures that suggest ways to cluster and prioritize groups of research areas mapping the research areas and topics to (1) specific DoD missions and required capabilities, (2) DoD research funding types required to support them, and (3) estimates of the relative starting points of the research. These structures can then be used to define one or more roadmaps that could lead to one or more ULS systems research programs or projects.

Recommendations

As a first step, we recommend the funding and establishment of a ULS System Research Startup Initiative, which over the course of the next two years would, among other things,

  • work with others to conduct new basic research in key areas;
  • foster the growth of a community of informed stakeholders and researchers; and
  • formulate and issue an initial Broad Agency Announcement (BAA) to attract researchers with proven expertise in the diverse set of disciplines (e.g., software engineering, economics, human factors, cognitive psychology, sociology, systems engineering, and business policy) that are collectively required to meet the challenge of ULS systems.

The United States needs a program that will fund the software research required to sustain ongoing transformations in national defense and achieve the DoD goal of information dominance. The key challenge is the decision to move forward. The ULS System Research Agenda presented in Ultra-Large-Scale Systems: The Software Challenge of the Future provides the starting point for the path ahead.