search icon-carat-right cmu-wordmark

Empirical Research Office

Created December 2017

The mission of the Empirical Research Office (ERO) is to improve the capability delivered for every dollar of U.S. Department of Defense (DoD) investment in software systems by ensuring that policy and program decision making is grounded in high-quality data and analysis.

Supporting the DoD’s Investment in Software Systems

The U.S. Department of Defense (DoD) is an organization that thrives on data and relies on software systems, yet there is a dearth of fact-based information in one very important area: software development and acquisition. Decision makers need credible and practical advice when it comes to creating policy related to the DoD's substantial investments in software systems. And, after these policies are implemented, they must be evaluated to find out whether they are working and how performance could be improved.

Our Work

Recognizing software's ever-increasing role in providing critical capabilities to the warfighter, the ERO provides the DoD with data-driven, empirically based approaches to

  • assess the state of the practice
  • evaluate methods and technologies
  • formulate guidance

To do this, the ERO engages in four broad classes of activities:

  1. Analysis—We provide decision makers with credible and practical advice for formulating policy, as well as measures and heuristics for assessing program health and evaluating emerging technologies. The ERO uses a fact-based approach to help determine which technologies are working and which are not and how improvements can be made in the development, acquisition, and sustainment of software-intensive systems.
  2. Data Collection—We maintain meaningful and descriptive data. To get the data needed to make relevant and actionable analytic work, the ERO engages directly with DoD programs and other appropriate sources and works with them on contractor data requirements.
  3. Tool and Infrastructure Development—We create appropriate infrastructure and tool support for managing and organizing the data and supporting low-cost analyses. This support includes making appropriate subsets of the data available for collaborative research projects with other FFRDCs and the larger research community.
  4. Community Engagement—We convene researchers and subject-matter experts to bring the best analytical methods and expertise to bear on DoD problems and elevate the quality of research and overall capability. The ERO leverages information from the broad research community to identify emerging trends and potentially disruptive new technologies.

Learn More

Department of Defense Software Factbook

July 11, 2017 Technical Report
Brad Clark, Christopher Miller, James McCurley, David Zubrow, Rhonda Brown, Mike Zuccher (No Affiliation)

In this report, the Software Engineering Institute has analyzed data related to DoD software projects and translated it into information that is frequently sought-after across the...

read

Panel: Software Sustainment - Continuous Engineering to Deliver Warfighter Capability

March 23, 2017 Presentation
Michael McLendon, Stephany Bellomo, Forrest Shull, John Stankowski (Office of the DASD for Maintenance Policy and Programs)

This technical panel focused on the DoD's software sustainment challenges and highlighted the key findings of the SEI's study of DoD software sustainment...

watch

Evaluation of Threat Modeling Methodologies

November 01, 2016 Presentation
Forrest Shull

The result of this work is a set of test principles that can help Programs select the most appropriate threat modeling...

watch