Technical Debt: I’ll Be Watching You
Every vow you break. Every smile you fake. Every claim you stake.
Technical debt is a metaphor and a storytelling device that can help explain the long-term consequences, costs, and risk introduced by prioritizing short-term cost avoidance over functionality, reliability, availability, and serviceability.
It’s a concept originally advanced by pioneers of the Agile software development movement to describe the tradeoffs between speeding development along and building workable and maintainable code.
Software systems are prone to the build-up of cruft — deficiencies in internal quality that make it harder than it would ideally be to modify and extend the system further. Technical debt was coined by Ward Cunningham in 1992, that, thinking of it as financial debt. The extra effort that it takes to add new features is the interest paid on the debt.
Ever since we started creating software products, we have been grappling with this issue under other names: software maintenance, software evolution, software aging, software decay, software system reengineering, and so on.
Technical debt is pervasive: It affects all aspects of software engineering, from requirements handling to design, code writing, the tools used for analyzing and modifying code, and deployment to the user base. The friction caused by technical debt is even apparent in the management of software development organizations, in the social aspect of software engineering.
Not always bad code
The original definitions of technical debt and the wide use of the term could lead us to think of it as simply bad code quality. Using terms with negative connotations — such as quick-and-dirty, shortcuts, bad design choices, death by a thousand cuts, and so on — amplifies this impression.
Low internal code quality is effectively a kind of technical debt — maybe the prevalent kind in the technical debt landscape. Tools including static code analyzers assist in identifying problems with low internal quality and related issues with documentation and testing.
However, as Steve McConnell, Martin Fowler, and others have pointed out, there are also deliberate, intentional, strategic decisions at the level of the architecture of the system or the choice of technologies that are made for an immediate gain, usually to reduce time to market. These choices also create technical debt, and they are all not related to bad code quality at all.
Technical debt landscape
It manifests in two main ways: difficulty and additional cost in evolving the system (that is, adding new functionality) or maintaining the system (that is, keeping the system running when the technical environment changes). But concretely, opening the box and looking at technical debt at the software level reveals that it takes many different forms.
When a system incurs technical debt at the source code level, the debt tends to hinder maintainability so that it will be hard to make corrections to the system when needed.
Other technical debt items are more encompassing and pervasive. They involve choices about the structure or the architecture of the system: choice of platform, middleware, technologies for communication, user interface, or data persistency.
For these elements, the principal and the interest are often higher than for technical debt in the code.
When a system incurs technical debt at the architectural level, the debt tends to hinder evolvability: The system will be hard to extend to new features, with their functional and quality attribute requirements, such as scaling to a larger number of users, processing different kinds of data, and the like
Finally, some technical debt items are associated not with the code of the product but with the code of other closely related artifacts in software production: build scripts, test suites, or deployment infrastructure.
Defects and low external quality — such as poor performance, a cumbersome user interface, instability, and security holes — are not technical debt. They are just poor external quality; the system is not operating properly, and its problems must be addressed.
However, the poor quality may be a consequence of technical debt.
Causes of technical debt
- Size: The size of the system is by far the greatest factor because it drives the size of the team, the number of teams, the need for communication and coordination between teams, the impact of change, and more. Size is often related to complexity. The larger the system, the more technical debt it can accumulate.
- Architecture: Many key architectural decisions are made in the first few days of development, such as choices related to middleware, operating systems, and programming languages. These choices may be based on what the developers are familiar with and their gut feelings rather than a careful analysis of long-term system consequences.
- Business model: What is the money flow? How is the project funded? Are you developing an internal system, a commercial product, or a component of a large system involving many different parties? Financial considerations are a key factor in incurring technical debt or deciding to remediate technical debt.
- Team distribution: Team distribution is often linked to the size of a project. Distributed teams increase the need for explicit communication and coordination of decisions as well as stable interfaces between the software components that they are responsible for. Communication issues and organizational silos contribute to the accumulation of technical debt, especially at the architectural level.
- Rate of change: Though agile methods are all for embracing change, not all systems experience a rapid pace of change in their environment. Many projects have very stable requirements definitions. How stable is your business environment, and how many risks and unknowns are you facing? The volatility of the requirements will increase the propensity of the team to incur technical debt.
Types of technical debt
- Code — Complicated code, cut and paste code, and tight coupling.
- Architectural — Missing and/or combining layers (e.g. business, service, presentation, etc.)
- Tests — Missing unit, automated integration, or improper test data and/or testing environment.
- Knowledge — Missing documentation; dead code — features that are obsolete.
- Technological — Progress of new technologies that have been missed.
Ways to manage growth of technical debt
Ultimately, technical debt is a technical risk, which in turn is a business risk that must be accepted or mitigated. To make good decisions on risk mitigation, you need to understand the size of the risk and the options available to address it.
Avoid new tech debt:
- Fix project closure and release processes
- Improve architectural practices
- Proactively budget for technology replacement
- Address smaller issues iteratively through the backlog
Pay back existing tech debt:
- Replace or retire legacy systems
- Refactor or rebuild poorly written non-standard code
- Refresh aging equipment
- Rehost systems or migrate to a new environment
You’ll always have technical debt
Technical debt is a byproduct of the relentless pace of technological change and the realities of software development. It can’t be avoided. At the same time, you have to manage how much technical debt you carry. If you don’t keep technical debt under control, your “interest payments” will prevent you from delivering new features and quality, timely products from to your users
You know the song. You know the lyrics. You know the artist.
