Managing Technical Debt with Data-Driven Analysis
Created September 2017 • Updated February 2022
Almost every software project carries some technical debt, but its causes and consequences can be elusive. We develop tools and techniques that identify technical debt and provide a complete view of the debt that you need to manage.
Technical Debt: A Compounding Problem
Technical debt is a term that conceptualizes the tradeoff between the short-term benefits of rapid delivery and the long-term value of developing a software system that is easy to evolve, modify, repair, and sustain. Like financial debt, technical debt can be a burden or an investment. Technical debt can be a burden when it is taken on unintentionally without a solid plan to manage it. Technical debt can also be part of an intentional investment strategy that speeds up development … as long as you have a plan to pay it back before the interest swamps your principal.
The U.S. Department of Defense (DoD) must constantly manage the dual challenges of budget constraints and the need to accelerate capability delivery. These challenges have encouraged the DoD to adopt incremental approaches to software development and to shift from the acquisition of new systems to the more cost-effective evolution and sustainment of existing systems. In this context, accumulated design and implementation decisions made for expediency, in the absence of structural quality requirements or without due consideration for sustainment and evolution, often result in systems that become prohibitively expensive to maintain or extend. In other words, these systems have a lot of technical debt.
Current tools for quantifying debt focus primarily on code quality and implementation decisions, such as complexity and cyclicity metrics. Our research has shown that key contributors to technical debt are design and architectural decisions that are not managed in terms of their consequences for system quality and rework. Existing tools cannot detect technical debt that results from design and architectural decisions. Moreover, the DoD and industry have started to invest heavily in model-driven and compositional software development approaches that existing technical debt identification techniques do not address. These types of debt are harder to recognize and occur earlier in system development, so the consequences can accrue for longer periods of time before they become visible.
Our Solution: Measure It and Manage It
The SEI is making software more sustainable through a comprehensive approach to managing technical debt. We are developing tools and techniques for uncovering technical debt that integrate data from multiple commonly available sources to reveal problematic decisions and quantify their consequences in a repeatable and reliable way. Together, the tools and techniques provide a complete view of the technical debt that you need to manage.
Our tooling approach combines existing code-oriented analysis tools with techniques that uncover and analyze architectural abstractions to reveal the structural causes of technical debt. It provides a more accurate scope for technical debt that exists in the system and assists both ongoing development and difficult sustainment decisions such as how to balance system improvements, early delivery, and upgrades.
Our Approach: Automating Technical Debt Analysis Through Software Analytics
We combine techniques from machine learning, refactoring and code analysis, and data mining with multiple data sources to describe technical debt items that identify problematic design issues.
- Create a technical debt classifier: Apply topic-modeling algorithms to issue tracker data to extract topics related to rework; extract categories of design related to technical debt.
- Correlate analysis rules with technical debt topics: Identify recurring design concepts and their mappings to code analysis rules; run code analyzers to detect quality violations to identify candidate technical debt items.
- Consolidate technical debt items: Run criteria for consolidations, and extract additional affected files with related violations.
- Rank technical debt items: Identify defects, change, and bug churn and locations in the code base that require changes; run causal analysis, and create an initial ranking.
This project will deliver a suite of tools and techniques for technical debt analytics, including
- a prototype tool that demonstrates clustering and ranking of technical debt items
- an extraction technique and a description template for technical debt items
- data set exemplars for organizations and researchers
December 15, 2019 Blog Post
Technical debt communicates the tradeoff between the short-term benefits of rapid delivery and the long-term value of developing a software system that is easy to evolve, modify, repair, and sustain. Like financial debt, technical debt can be a burden or an investment. It can be a burden when it is...read
June 24, 2019 White Paper
This study introduces (1) a dataset of expert labels of technical debt in developer comments and (2) a classifier trained on those labels.read
May 12, 2019 Blog Post
If you participate in the development of software, the chances are good that you have experienced the consequences of technical...read
April 19, 2019 Book
This book is for every software professional who wants to accelerate innovation in existing systems or build new systems that will be easier to maintain and evolve.read
April 30, 2017 Blog Post
Software design problems, often the result of optimizing for delivery speed, are a critical part of long-term software costs. Automatically detecting such design problems is a high priority for software practitioners. Software quality tools aim to automatically detect violations of common software...read
August 28, 2016 Blog Post
Software engineers increasingly recognize technical debt as a problem they care about, but they lack methods and tools to help them strategically plan, track, and pay down debt. The concept provides a vocabulary to engage researchers from a practice point of view, but they often lack an empirical...read
May 16, 2016 Conference Paper
This paper reports on a study of issues from issue trackers to identify technical debt and present an approach for reporting technical debt in issue trackers.read
April 05, 2016 Conference Paper
This paper presents an in-depth study of a safety-critical system that underwent major changes as a result of missed architectural dependencies.read