search menu icon-carat-right cmu-wordmark

Managing Technical Debt with Data-Driven Analysis

Created September 2017

Almost every software project carries some technical debt, but its causes and consequences can be elusive. We develop tools and techniques that identify technical debt and provide a complete view of the debt that you need to manage.

Technical Debt: A Compounding Problem

Technical debt is a term that conceptualizes the tradeoff between the short-term benefits of rapid delivery and the long-term value of developing a software system that is easy to evolve, modify, repair, and sustain. Like financial debt, technical debt can be a burden or an investment. Technical debt can be a burden when it is taken on unintentionally without a solid plan to manage it. Technical debt can also be part of an intentional investment strategy that speeds up development…as long as you have a plan to pay it back before the interest swamps your principal.

The Department of Defense (DoD) must constantly manage the dual challenges of budget constraints and the need to accelerate capability delivery. These challenges have encouraged the DoD to adopt incremental approaches to software development and to shift from the acquisition of new systems to the more cost-effective evolution and sustainment of existing systems. In this context, accumulated design and implementation decisions made for expediency, in the absence of structural quality requirements or without due consideration for sustainment and evolution, often result in systems that become prohibitively expensive to maintain or extend. In other words, these systems have a lot of technical debt.

Current tools for quantifying debt focus primarily on code quality and implementation decisions, such as complexity and cyclicity metrics. Our research has shown that key contributors to technical debt are design and architectural decisions that are not managed in terms of their consequences for system quality and rework. Existing tools cannot detect technical debt that results from design and architectural decisions. Moreover, the DoD and industry have started to invest heavily in model-driven and compositional software development approaches that existing technical debt identification techniques do not address. These types of debt are harder to recognize and occur earlier in system development, so the consequences can accrue for longer periods of time before they become visible.

Our Solution: Measure It and Manage It

The SEI is making software more sustainable through a comprehensive approach to managing technical debt. We are developing tools and techniques for uncovering technical debt that integrate data from multiple commonly available sources to reveal problematic decisions and quantify their consequences in a repeatable and reliable way. Together, the tools and techniques provide a complete view of the technical debt that you need to manage.

Our tooling approach combines existing code-oriented analysis tools with techniques that uncover and analyze architectural abstractions to reveal the structural causes of technical debt. It provides a more accurate scope for technical debt that exists in the system and assists both ongoing development and difficult sustainment decisions such as how to balance system improvements, early delivery, and upgrades.

Our Approach: Automating Technical Debt Analysis Through Software Analytics

We combine techniques from machine learning, refactoring and code analysis, and data mining with multiple data sources to describe technical debt items that identify problematic design issues.

  1. Create a technical debt classifier: Apply topic-modeling algorithms to issue tracker data to extract topics related to rework; extract categories of design related to technical debt.
  2. Correlate analysis rules with technical debt topics: Identify recurring design concepts and their mappings to code analysis rules; run code analyzers to detect quality violations to identify candidate technical debt items.
  3. Consolidate technical debt items: Run criteria for consolidations, and extract additional affected files with related violations.
  4. Rank technical debt items: Identify defects, change, and bug churn and locations in the code base that require changes; run causal analysis, and create an initial ranking.

This project will deliver a suite of tools and techniques for technical debt analytics, including

  • a prototype tool that demonstrates clustering and ranking of technical debt items
  • an extraction technique and a description template for technical debt items
  • data set exemplars for organizations and researchers

Learn More

Automating Design Analysis

Automating Design Analysis

May 01, 2017 Blog Post
Neil Ernst

Software design problems, often the result of optimizing for delivery speed, are a critical part of long-term software costs. Automatically detecting such design problems is a high priority for software practitioners. Software quality tools aim to automatically detect violations of...

read
Got Technical Debt? Track Technical Debt to Improve Your Development Practices

Got Technical Debt? Track Technical Debt to Improve Your Development Practices

June 27, 2016 Blog Post
Stephany Bellomo

What is technical debt? Why identify technical debt? Shouldn't it be captured as defects and bugs? Concretely communicating technical debt and its consequences is of interest to both researchers and software engineers. Without validated tools and techniques to achieve this...

read
Got Technical Debt? Surfacing Elusive Technical Debt in Issue Trackers

Got Technical Debt? Surfacing Elusive Technical Debt in Issue Trackers

May 16, 2016 Conference Paper
Stephany BellomoRobert NordIpek Ozkaya

This paper reports on a study of issues from issue trackers to identify technical debt and present an approach for reporting technical debt in issue trackers.

read
Missed Architectural Dependencies: The Elephant in the Room

Missed Architectural Dependencies: The Elephant in the Room

April 05, 2016 Conference Paper
Robert NordRaghvinder SangwanJulien Delange

This paper presents an in-depth study of a safety-critical system that underwent major changes as a result of missed architectural dependencies.

read
Reducing Friction in Software Development

Reducing Friction in Software Development

January 04, 2016 Article
Paris Avgeriou (University of Groningen, The Netherlands)Philippe KruchtenRobert Nord

Getting ahead of the software quality and innovation curve will involve establishing technical-debt management as a core software engineering practice.

read
Measure It? Manage It? Ignore It? Software Practitioners and Technical Debt

Measure It? Manage It? Ignore It? Software Practitioners and Technical Debt

August 30, 2015 Conference Paper
Neil ErnstStephany BellomoIpek Ozkaya

This paper reports on a survey of 1,831 software engineers and architects, and follow-up interviews of seven software engineers, to determine the most important sources of technical debt.

read