Meeting the Challenge of Ultra-Large-Scale (ULS) Systems



Bill Pollak

This library item is related to the following area(s) of work:

Ultra-Large-Scale Systems

This article was originally published in News at SEI on: April 1, 2005

“Given the issues with today's software engineering, how can we build systems of the future that are likely to have billions of lines of code?”

Claude M. Bolton, Jr., assistant secretary of the Army (Acquisition, Logistics, and Technology), posed this question to the SEI in 2004. The question led the SEI to conduct a research study in ultra-large-scale (ULS) systems: systems that exceed some critical limit of today’s software engineering technology. Although Assistant Secretary Bolton initially defined the challenge of ULS systems in terms of billions of lines of code, size is only one limit. Others include unboundedness, continuous requirements evolution, and continuous operation.

The intended outcome of the research study is the creation of a technology roadmap for ULS systems and a collaborative research network that works toward solving the ULS problem. “Our soldiers depend on software and will depend more on software in the future,” says Assistant Secretary Bolton. “The Army’s success depends on software and the software industry. We need to build reliable ULS systems to meet the needs of the military and society at large.”

Software, says Bolton, is the chief enabler of an Army transformation that emphasizes information superiority. “Software makes possible increased situational awareness by providing sensors into networks that allow commanders and soldiers to see first, act first, and act decisively,” he says. But the Army’s demands for software are rapidly outpacing its ability to manage software acquisition. The Army’s Future Combat Systems (FCS), for which it plans to develop and acquire more than 33 million lines of code (LOC), is already challenging current software engineering capabilities. “We need better tools to meet future challenges,” says Bolton, “and neither industry nor government is working on how to do things light-years faster and cheaper. How can future systems be built reliably if we can’t even get today’s systems right?”

Peter Freeman, assistant director of the National Science Foundation for Computer and Information Science and Engineering, concurs. “Our concepts for new systems seem larger than the reality of developing them,” he says. The problem, says Freeman, is that the ideas and concepts needed to meet future challenges may be outside the discipline of software engineering as it is currently defined and understood. “We as a field need to think deeply and carefully to characterize the nature of the problem and the nature of the material with which we work—what is software, what are systems, and what are components? How do we characterize the relationships between these, and how do we characterize the dynamic behavior of systems? We don’t have the intellectual basis in place for science and design to do the things that Assistant Secretary Bolton envisions. So we need new ideas.”

The study that the SEI is conducting is intended to generate such new ideas. It brings together software experts with a wide range of knowledge and expertise representing the SEI, Carnegie Mellon University, and several other institutions and organizations (see sidebar). The initial intended outcome of the study is a program definition including

  • a technology roadmap
  • defined areas of investigation
  • key organizations to be involved
  • funding required

The team agreed that incremental improvement of current approaches will not suffice to solve the ULS system problem. “If our ways of thinking are behind the problems that we face today,” says team member Jack Whalen of Palo Alto Research Center, “we need to seriously rethink how we conceptualize the things that we take for granted.”

Linda Northrop, director of the SEI Product Line Systems Program and leader of the ULS project at the SEI, agrees. “This is precisely what we’ve been charged to do,” she says. “This is the kind of thinking that Mr. Bolton wants to encourage. To make significant progress in the size and complexity of systems that can be built and deployed successfully, we require a culture shift. Our task is to identify the kinds of research that will effect such a culture shift.”

The team held its first meeting on August 16-19 at the SEI's main office in Pittsburgh. At the meeting, participants discussed and identified the organizing elements for a ULS technology roadmap: ULS system characteristics, technical challenge areas, and breakthrough research areas.

“One of the questions we struggled with when we first started is how to define ULS system characteristics,” says Mark Klein, an SEI member of the team. “We rejected the simple notion that ‘ultra’ applied only to the number of lines of code. Somehow as the system grows along various dimensions, conventional wisdom no longer applies. These systems must always run. They are seemingly ubiquitous. They will never be error free. No small number of operators can control them, and the number of operators required would require too much coordination. The distinction between runtime and design is blurred. These are examples of what we mean by ‘ULS system characteristics’.”

Technical challenge areas are a group of technical capabilities that seem likely to be difficult for ULS systems.

Breakthrough research areas are those that are likely to make the development of ULS systems more tractable—either a new technical approach or a breakthrough in an existing approach needed to address one or more of the ULS technical challenge areas. Three research-study subteams formed at the meeting are currently working on defining breakthrough research areas related to the defined technical challenges.

The next meeting of the team will be held Nov 29-30, again in Pittsburgh. The SEI plans to complete the effort on April 30, 2006, when a full report will be available.

Invited speakers at the August 2005 ULS System Meeting meeting included

  • Assistant Secretary Claude M. Bolton, (on video)
  • Peter Freeman, National Science Foundation
  • David Emery, DSCI
  • Bruce Krogh, School of Electrical and Computer Engineering, Carnegie Mellon University 

Members of the external expert panel are

  • Gregory Abowd, Georgia Institute of Technology
  • Carliss Baldwin, Harvard Business School
  • Robert Balzer, Teknowledge Corporation
  • Richard Gabriel, Sun Microsystems, Inc.
  • Gregor Kiczales, University of British Columbia
  • John Lehoczky, Carnegie Mellon University
  • Ali Mili, New Jersey Institute of Technology
  • Peter Neumann, SRI International
  • Douglas Schmidt, Vanderbilt University
  • Mary Shaw, Carnegie Mellon University
  • Daniel Siewiorek, Carnegie Mellon University
  • Kevin Sullivan, University of Virginia
  • Jack Whalen, Palo Alto Research Center              
SEI members of the ULS team are
  • Linda Northrop
  • Peter Feiler
  • John Goodenough
  • Rick Kazman
  • Mark Klein
  • Rick Linger
  • Tom Longstaff
  • Jim Over
  • Mark Pleszkoch
  • Kurt Wallnau

Find Us Here

Find us on Youtube  Find us on LinkedIn  Find us on twitter  Find us on Facebook

Share This Page

Share on Facebook  Send to your Twitter page  Save to  Save to LinkedIn  Digg this  Stumble this page.  Add to Technorati favorites  Save this page on your Google Home Page 

For more information

Contact Us


Help us improve

Visitor feedback helps us continually improve our site.

Please tell us what you
think with this short
(< 5 minute) survey.