Choosing an easy solution to solve a problem always seems tempting. But, the additional cost of rework that comes your way if things do not work out can get exhausting. In terms of Agile software development methodology, an easy overall solution is often chosen to facilitate a faster time to market while compromising quality code, resulting in an additional cost of rework later, i.e., technical debt.
Technical debt in software development projects is common as organizations prioritize faster time to market over quality deliveries.
It is similar to postponing paying your monthly bills. By doing so, the bill amount keeps adding up along with interest due to exceeding due dates. So, when you pay your bill after six months or so, it gets challenging to repay it.
Technical debt consumes 40% of development time and results in higher operational expenses. — Appian
Here’s a complete guide that will help you understand the forwards and backward of technical debt in Agile software development.
What is Technical Debt in Agile?
“We need to launch the innovative feature within a month,” said the product owner.
The developers accelerate the work on the new proposed feature in the planned sprint cycles.
But, in the process — delay bug fixtures in the developed user stories and write messy and disconnected code.
The upshot: The team manages to launch the new feature on time but at the cost of over accumulated bugs across sprint cycles and uniterable code. Thus, leaving them with unwanted technical debt.
We can thus, define technical debt in Agile as:
Technical debt is the implied cost of additional work related to — choosing a shortcut to software development and ignoring the pending bug fixtures (in developed features) to work on new features with the intent to hit the launch dates.
In simple words, it is about preferring incremental development while ignoring iterative development in Agile. However, this practice only offers short-term benefits as it compromises on long-term value.
Although the product backlog prioritizes the changes needed to the already developed features in the consecutive sprint cycles, in reality, they are often left unattended.
However, there is another interpretation that implies:
Technical debt is the implied cost of additional work related to — writing messy or unrefined code when in a hurry to launch, which will get challenging to refactor later. This interpretation is directly linked to software architecture — building organized code for making refactoring easy.
Technical Debt Example
Let us say you send the developed user story to the Agile testing team for review. Now, there are two scenarios at play:
- The testing team reports some bugs with respect to the developed feature. Your team ignores those fixtures and moves ahead with the next user story in the queue. Thus, resulting in technical debt, the extra cost of fixing over layered bugs in one go later
- The Agile team approves the feature. It is then integrated, delivered, and deployed into the live environment. Sometime later, when a change is required, it becomes challenging for developers to append the new code into the existing one because of the code’s original messy structure. Thus, resulting in technical debt
What are the Different Types of Technical Debt?
Martin Fowler introduces different types of tech debt in the form of a quadrant, which he refers to as “Technical Debt Quadrant.” Let us have a look.
Here’s what each of them implies:
This can be linked to the intentional injection of technical debt in the software development process. The two interpretation of reckless and prudent apply:
- Reckless: This type of debt is the result of bad software architecture. In this case, the development team believes that they do not have enough time to build organized code as they focus more on shipping faster
- Prudent: In prudent debt, the technical debt is small and manageable. Thus, the tech debt can be avoided as it is mostly related to the code that rarely needs modification
This can be linked to the unintentional injection of technical debt in the software development process. The two interpretation of reckless and prudent apply:
- Reckless: In reckless inadvertent debt, the development team lacks knowledge and experience around best Agile practices. This might be a case for organizations that are halfway on the road to Agile transformation. In turn, they unintentionally build up on debt, which becomes difficult to handle later
- Prudent: Once the feature is done and delivered, the team realizes that they could have done better (credits to their learnings). This is referred to as prudent inadvertent debt
What are the Causes of Technical Debt?
Before you strategize around reducing technical debt, it is essential to understand its origin.
Here are some common reasons behind recurring technical debt:
1. Prioritizing Faster Time to Market
Organizations into product development believe in the “just ship it,” philosophy.
In the process, they develop pressure on the Agile development team, who then choose to ignore the incoming bugs across past sprint cycles and instead move ahead with developing messy codes. This, in turn, leads to increased technical debt as the existing inconsistencies in the code are left unattended.
2. Misinterpreted Definition of Done
In Agile, “Definition of Done,” implies — a pre-decided list of tasks that need to be completed before considering a user story to be complete. Agile teams often consider a user story to be done once it is passed on to the QA testing team or once the feature is launched in the market.
In reality, the changes based on testing reports/user feedback are not part of the definition of done, which leads to increasing technical debts. In other words, prioritizing project mindset over product mindset leads to tech debt when iterating through the Agile lifecycle.
3. Obsession with Product-Market Fit
If a product can solve a user pain-point and is accepted by the customers, it is considered a product-market fit. And, achieving this product-market fit is all that organizations care about.
Messy code and unresolved issues are ignored as far as product-market fit is in place. However, organizations fail to realize that this ignorance can cost much in the long run with piling up technical debts, leading to performance issues and, thus, disrupted PMF.
4. Feature Creep
Feature creep is the capability of software requirements to increase at any point in the software development process. This unpredictability leads to a change in plans and sprint prioritizations.
We respect your privacy. Your information is safe.
More often, to accommodate the newer requirements into the product backlog, the bug fixtures take a backseat and are left for tomorrow. Hence, the increasing technical debt.
5. Exceeding WIP (Work in Progress) Limits
WIP limit refers to the maximum amount of work that can exist in the status of a workflow. Ideal WIP limits can be “number of team members +1 to start with.”
If the WIP limit increases for any reason, it can lead to confusion with priorities and work overload (irrespective of its urgency). This, in turn, affects productivity as the development team works on completing tasks without worrying about quality delivery. Hence, the technical debt.
6. Unmanageable Growth Velocity
Sometimes the velocity of new feature requests is so high that teams lose focus from technical debt. Take the example of Facebook. They get new feature requests so often that they have their development teams packed up with serving those requests.
This, in turn, draws them away from the increasing technical debt, which, thus, keeps growing and reaches a stage where it gets unmanageable.
How to Identify Technical Debt?
Here are 12 signs that will help you identify the presence of technical debt in your existing projects. These include:
- The product backlog is expanding day by day, while the productivity seems to deteriorate
- Upon analyzing, you realize that the WIP (Work in Progress) limit has gone up to the number of team members plus “n.” The Agile team is all packed up and has low bandwidth for working on amendments
- The frequency of bugs increases. This could be the result of the development team struggling with managing the variable product backlog and pressure from the product manager
- The source code is pretty long and seems to exceed the expected space it covers. This could be because of the messy code and antipatterns injected in the code
- If adding a new feature or making amendments to an existing feature means that you need to rewrite a large chunk of existing source code
- Technical debt is quite evident if unit testing a module is painstaking and complicated
- The result of the quantified cost of delay (the implied cost of delaying the launch by one week, one month, or a year) is much higher than expected
- The frequency of hotfixes (code that helps fix a bug) seems to on a rise
- The continuous increase in lead time is noticed along with the decrease in velocity, which puts you in a tight spot
- Increase in write-only code,i.e., the code that is hard to read through
- There is a technical debt if the burndown chart shows a lot of pending work in the backlog
- If there exists gold plating, i.e., unnecessary items in the product backlog — messy code is the result, which, in turn, leads to technical debt
What is the Cost of Technical Debt?
If you come to analyze the cost of technical debt, you’ll know why it is essential to identify and manage.
The Cost of ‘Technical Debt’ Per Average Business Application Exceeds $1 Million. — IT Business Edge
Here’s what you will lose if technical debt is ignored:
- Delayed bug fixtures and new user story launches, which will lead to loss of customer interest
- You will have to recruit more staff for identifying and managing technical debt, which might take even longer to get back on track
- Software system outages as a result of continuous code modification for bug fixture or feature additions
- Continued performance issues that negatively impact the responsiveness of the product
Is Technical Debt Bad?
Any type of debt is bad. For example, when you have incurred debt in your name, you will have to repay it sooner or later. There is no running away from it!
Similarly, carrying a technical debt might seem worth the risk as you get an early launch and more features, but you’ll eventually have to pay it back. But if you keep delaying the payback, the impact of technical debt will be adverse as messy becomes messier, and product backlog becomes unmanageable.
Talking with experience, we believe that zero technical debt is a myth.
Although there is always some technical debt, the effort should be to keep trimming its branches to limit its spread.
Let us take an example here:
The Good Story:
- You launch the MVP on 1st Feb 2021. You disrupted the market with an early launch — you gain a competitive advantage
- Meanwhile, the Agile team works on adding some new features, i.e., the nice-to-haves, which they prioritized post the MVP development and launch
- The initial feedback based on the MVP comes in within a month
- A backlog grooming session is held to prioritize the tasks, which includes both — the new features + the refinements to existing features
The bad Story:
- The Agile team’s flow of work is disturbed as they now have to focus on improving the existing features as well
- However, more often, the team focuses more on building new features and delays improving the existing features
- Even if the team chooses to improve the existing features — iterating through the source code becomes impossible. This is because of the poorly written code to rush up the launch
- Newer feature launches get delayed, and so do the refactoring activities — the upshot of it — a bad customer experience and increased churn rates
When to Accept Technical Debt?
Some amount of technical debt always exists, which is why it is recommended never to obsess over eliminating it entirely.
Accepting technical debt is sometimes better than wasting your resources fixing it. But, how do you know — when to accept technical debt?
It would be best if you accept technical debt in the following scenarios:
- You are onto small-scale product development where refactoring is easy and quick
- When you are building an MVP with minimal features. In this case, you can afford to wait for initial feedback to validate product-market fit before choosing to refactor
- You have an experienced team that understands technical debt and is ready to go the extra mile to manage it
- You are trying your hands at new product development that includes an out-of-the-box feature that solves a user problem. In this case, launching the MVP with this must-have feature should gain focus (while you can ignore tech debt until its market acceptance)
How to Measure Technical Debt?
Quantifying technical debt is the first step that the product development team takes before getting on with managing it.
Several technical debt tools can provide an estimation of the incurred debt. Here’s a table that gives you an overview of each of them:
|1. SonarQube||Open community tool to help developers write cleaner and organized code.
The tool has a quality gate metric that qualifies your code against code smells, debts, vulnerabilities, and bugs.
|2.Coverity||Focuses on delivering software quality.
Helps identify resource leaks, memory corruptions, error handling issues, and more.
|3. Checkstyle||Helps scan and analyze Java code with respect to coding standards.
Checks code layout, formatting issues, overall code layout, and class layout issues.
Another method for measuring technical debt is through the “Technical Debt Ratio” (TDR). It is a complicated process when compared to using automated tools that facilitate automated calculations.
Technical debt metrics that you need to take into account while calculating the ratio:
- Development cost calculated in hours — time taken to develop a feature/product
- Lines of code (LOC) — the total number of code lines
- Cost per line (CPL) in hours — time to write a line of code
- Remediation cost in hours — time to make a repair
Here is the formula for calculating the technical debt ratio:
Technical debt ratio = (Remediation Cost / Development Cost) X 100
Development Cost = Cost per line (CPL) X Lines of code (LOC)
Prefer maintaining a low TDR percentage at all stages. A value that is less than 5% is relatively considerable for Agile teams. However, this value depends on the project size and scope, and every team has defined ballpark TDR values considered as standards.
Who is Responsible for Technical Debt in Scrum?
Holding a particular team or an individual accountable for technical debt kills the foundation of Agile, which focuses on collaboration and cross-functional setup.
So, the right answer to who is responsible for technical debt is — Everyone!
Every team and each individual is somehow responsible for incurring technical debts.
- Development Team: when setting WIP limits, the development team should voice their opinion on how much work is acceptable. If they don’t talk initially, the cost of technical debt increases
- Product Owner: prioritizes tasks for the Agile development team. If the product owner creates aggressive goals for the team, they either follow shortcuts or write messy codes to comply with work commitments
- Top Management: if the top management forces a culture where deliveries are given precedence over quality work, the technical debt is likely to increase
- Scrum Master: is responsible for identifying and removing the obstacles restricting quality software development. If the scrum master fails to see the development team’s struggles w.r.t. managing quality deliveries, technical debt can get worse than expected
How to Manage Technical Debt?
If the Agile team keeps borrowing the technical debt, it might become impossible to repay it later. This is why following best practices for reducing technical debt should take precedence.
Here are some proven techniques for technical debt management:
1. Change in Organizational Culture
Build an open organizational culture where teams can talk about the challenges in achieving their goals.
Addressing the elephant in the room, i.e., the reasons behind technical debt will help strategize around managing the workflow and build realistic timelines for completing tasks at hand.
2. Take Daily Standups Seriously
The teams should be able to talk about the challenges that they are facing without hesitation. However, this happens to be a common Agile antipattern in distributed teams.
Likewise, the scrum master’s responsibility is to ensure that the highlighted challenges are addressed immediately, such as meeting unrealistic goals in a limited time.
3. Balance Sprints
Keep a day in a week (for small scale projects) or an entire week (for large scale projects) for managing the pending tasks. For example, when the team is done with building a new user story, they should be given a day in a week just for bug fixtures in existing features.
This way, they will be able to work effectively on the new user story while also paying the technical debt related to completed user stories.
We respect your privacy. Your information is safe.
4. Collective Ownership
Collective ownership implies that every development team member can make changes to the code to develop a new feature, complete a pending task, or implement a bug fixture.
By implementing collective ownership, the Agile team can reduce the technical debt – as the development team can distribute and manage work among themselves to build new features and make changes to existing ones simultaneously.
5. Prefer Quality over Quick and Dirty
Quality code should be preferred, even if it takes longer to get it done. Do not promote a work culture where the speed of development is a measure of productivity. The real measure of productivity should be quality at any given time.
If the team creates messy code to facilitate fast shipping, maintaining it becomes similar to grasping at straws with no improvement. So, ship fast but not by compromising on the software architecture.
6. Introduce CI/CD into the Process
Implementing CI/CD is synonymous with introducing automation testing in the development process. Even if the Agile team suffers from low bandwidth to look into bugs, the automation testing process will make things better.
CI/CD will help in running automated tests for the developed user stories, which, in turn, will help report the bugs faster. The development team can fix the bugs in the following sprint cycle (while they are already in the flow of working on the developed user story). Thus, reducing the probability of existing bugs in delivered and deployed user stories.
7. Manage the Load
If the developers are all stocked up and have aggressive timelines to meet the launch date, they are likely to choose an easy way out — sometimes in the form of messy codes. In this case, the developers only do their job for the sake of completing it while compromising on quality.
So, unload your developers and set realistic WIP limits for them. If they have less mess to handle in the product backlog, the code quality is less likely to be jeopardized, which further implies less technical debt down the line.
8. Daily Refactoring vs Sustainable Refactoring
If you are working on a small project, regular refactoring should be standard practice. The developers should be given enough bandwidth in their daily schedule to fix bugs. This is daily refactoring, which leaves the development team with minimal debt at the end of the software development lifecycle.
The other type of refactoring is sustainable refactoring, which can be another practice to follow. Here the team focuses heavily on developing new features (80%) and refactoring the written code on a small scale (20%). This allows eliminating technical debt sustainably.
9. Rely on Microservices Architecture
Microservices architecture introduces loose coupling where a complete feature or a module is independent of other features/modules. In case a particular feature suffers downtime, the other features still manage to function due to loose coupling.
If there is a need to call out another feature, it is done with APIs. Relying on microservices is a good way to reduce performance issues occurring due to technical debt because — when the team is busy refactoring one part of the software system, the other parts continue to operate normally.
Technical debt is a real concern for software development companies, no matter how small or large the project. Organizations always seem to run behind serving new feature requests and shipping them faster, which, in turn, lead to bad code quality and skipped refactoring.
The opportunity cost of shipping faster over maintaining a high quality of the code can sometimes get higher compared to profits incurred. This is why paying off technical debt is essential, like any financial debt. This is one important lesson of effective product development.
However, it is ok to accept some amount of technical debt. But, letting it grow to the extent that it becomes unmanageable should be avoided.
Here’s the secret — it is both about building the right product and building the product right.