Applying Causal Learning to Improve Software Cost Estimation and Project Control
Created March 2021
Software plays a major role for the Department of Defense (DoD), and its importance for DoD systems is only increasing. As the DoD looks to acquire and develop new software systems, developing plans for the future depends on having an accurate analysis of software cost. Many cost estimates, however, are often inaccurate, causing programs to break their plans as well as their budgets. To improve cost estimation, SEI researchers are exploring how to apply causal learning and discovery methods as an alternative to traditional cost analysis. Causal-learning approaches show greater success in helping software development and acquisition programs stay on track to achieve their goals.
Inaccurate Estimates Break Budgets and Derail Planning
When planning software systems development or acquisition, the DoD has run into problems like increased costs and delays. The recent DoD Cost Estimating Guide emphasizes the importance of developing “cost estimates that support the planning, programming, budgeting, acquisition, and requirements generation processes.” To make sure programs can stay on track and on budget to deliver the software capabilities the DoD needs to plan for the future, accurate cost estimates are key.
Usually, traditional cost estimation focuses only on predicting costs. These estimation methods rely on tools such as statistical correlation, regression, and even complex machine-learning applications—all of which are effective at finding associations in data to make good predictions. What these methods fail to do, however, is to understand what is driving those costs and what causes them to change. If and when costs fluctuate, these methods don’t provide a good way to manage them or understand why.
The goal to improve planning is to develop a method that can find the causes that drive costs. When a method provides such insight, organizations can take back control to salvage their planning and avoid rising costs by changing policies or taking management action. To control project outcomes, we need to know which factors truly drive them so we can manage the issues that lead to late delivery, cost overruns, and performance gaps before they ruin projects.
Getting a Handle on Software Cost
At the SEI, we have been developing modeling tools based on causal learning to identify and evaluate the issues that lead to delays, broken budgets, and other poor outcomes. Causal learning focuses on studying the relationships between a cause and its possible effects so that we can understand how we arrive at certain outcomes and how we might control them better. Research has shown that causal models for cost estimation are superior to traditional statistical models because, by identifying causal factors, proactive control of project and task outcomes becomes possible.
Over a three-year period, SEI researchers have worked to help the DoD—especially the U.S. Army and USC—to apply causal learning to control costs in software development and sustainment. We also explored how to deal with sources of uncertainty in project planning and studied traditional cost estimation models. Thanks to this work, we have been able to identify causal factors in software and systems engineering that provide ways for controlling program costs. As our research in causal learning continues to evolve, it will also help organizations better negotiate software contracts, design more effective policies and incentives, and provide more actionable information for affordability reports.
During our work with project planning, we’ve also uncovered several pitfalls that many organizations face, including poor communications, unclear planning, and ineffective hiring practices. Many organizations are often unaware of these issues, and are therefore unable to mitigate them. The SEI can help your organization identify these kinds of issues, as well as improve planning and invest your resources more effectively to chart a clear path toward achieving your program’s objectives.
Learn More
How Can Causal Learning Help to Control Costs?
•Blog Post
This post describes how to apply causal-modeling algorithms and tools to project data to identify, measure, and test causality.
READApplying Causal Learning to Evaluate Large Language Models (LLMs)
•SEI Report
This report describes how the SEI applied causal discovery to LLM summarization, exemplifying a way to address bias when evaluating complex new technology.
ReadSEI Tool Helps Determine Causes of AI Bias, Improves AI Trust
•Newsletter
This SEI Bulletin newsletter was published on July 23, 2025.
ReadFrom Data to Performance: Understanding and Improving Your AI Model
•Podcast
Drift in data and concept, evolving edge cases, and emerging phenomena can undermine the correlations that AI classifiers rely on. In this podcast, SEI researchers discuss a new tool to help improve AI classifier performance.
ListenAI Robustness (AIR)
•Presentation
Linda Parker Gates and Dr. Nicholas Testa presented this project at the CMU SEI Research Review 2024.
Learn MoreAI Robustness (AIR) Tool
•Software
The AI Robustness (AIR) tool allows users to gauge AI/ML classifier performance with unprecedented confidence.
DownloadGetting Started with AIR
•Fact Sheet
Learn the benefits of being an Artificial Intelligence Robustness (AIR) partner at the SEI.
Learn MoreMeasuring AI Accuracy with the AI Robustness (AIR) Tool
•Blog Post
Understanding your artificial intelligence (AI) system’s predictions can be challenging. In this post, SEI researchers discuss a new tool to help improve AI classifier performance.
READ