The Power of Technical Debt
Lately, I have been in a number of discussions about Technical Debt and how it applies to project development. Overall, I think it's a very powerful tool that, if used wisely, can be a great asset to any team. It seems like the people that I've been talking to really don't agree, and see Technical Debt as a plague that should be eliminated at first sight. So, I figured I'd share my opinions, and see what you think...
What Is Technical Debt
I think the best way that I can explain what technical debt is, is to let someone else do it for me. Martin Fowler explains what technical debt ispretty clearly:
...doing things the quick and dirty way sets us up with a technical debt, which is similar to a financial debt. Like a financial debt, the technical debt incurs interest payments, which come in the form of the extra effort that we have to do in future development because of the quick and dirty design choice. We can choose to continue paying the interest, or we can pay down the principal by refactoring the quick and dirty design into the better design. Although it costs to pay down the principal, we gain by reduced interest payments in the future.
I really like the financial analogy, so let's use that as a starting point. Just like financial debt, technical debt comes in many forms. And it also can be very good, or very bad, depending on how and when it is used. And each form has a varying amount of risk, and manifests itself differently. So, let's look at four basic types of technical debt, equating it to the financialequivalent.

Payday Loan
The easiest and most obvious form of technical debt is basically the payday loan. You don't have enough money to pay your bills, so you take a loan against a future pay check that you're going to receive. In the programming world, this equates to not having enough time to finish the application properly before a deadline and choosing the quick and dirty method to get you to the finish line. In both cases, this is a really bad idea and you should only use it in a dire emergency.

The problem is that after you've taken the debt, you usually can't have enough time to pay back the debt after the deadline. That is because there is usually another deadline coming that you also won't be able to hit without taking debt. Remember, debt is never free, and always reduces your overall availability to work. So if you can't complete your task in the allotted time without taking on debt, you won't be able to complete the next one either. So you accumulate more debt. And it continues as a vicious cycle.

It's usually far better to bite the bullet and admit that you can't make the deadline. Remember, communication is key. It's better to present to the stakeholders that in order to make the deadline you'd have to accumulate a significant amount of debt, and that future deadlines would need to be pushed back to repay the debt. Don't be silent. Don't let it slide and hope to recover. Be open, and it usually won't be a problem... Remember:
If you don't have enough time to do it right, when will you have time to do it over?

Mortgage
This form of debt is also very common, but is a little less obvious than the payday loan. In the financial world, you would accrue a mortgage loan in order to make a major purchase that you want to spread out the payments for over a long time (usually a home). When developing, you'd take a mortgage when you only build a small part of a much larger system, because that's the only part you need right now. This is an extremely common and powerful concept, but if not understood carefully can really cause pain. Let's take an example of a real world problem that you would use this technique for.
Let's say we're building an application that is going to communicate with the Stack Exchange API. We could build a data mapper and domain logic to communicate with everything the API offers. But instead, we can choose to design our libraries to be able to talk with the full API, but only actually implement the methods we need at the current time. If we do this, we're taking a mortgage out against that library, as we are delaying a major payment, and spreading it out over time as needed.
The thing to realize here is that we're taking a risk by doing this. If we wind up needing the full API, we'll wind up repaying a lot of interest in that it will take longer doing itpiecemealinstead of all at once. And we may be further hurt because we were expecting something to work which wasn't fully implemented, causing future delays.

If used properly, this type of debt can be actually very good to accrue. In fact, sometimes it's actually more prudent to do this, as it can prevent you from writing code which you will never use. But like all debt, you need to keep track of it and not accrue too much...

Credit Card
In the financial world, a credit card can be used when you're not quite sure what you need, and as such the cost is not fully known at the time of opening the debt. The same thing happens when we're developing an application. We'll run into situations where we don't understand the problem well enough to build an appropriate solution. So instead of trying to build a perfect solution, we build a sub-optimal solution which lets us work on the more important parts of the problem.

By doing this, we're admitting that we don't quite have enough information to solve the problem well, so instead we put in a temporary solution which lets us understand the problem better. Personally, I use this paradigm all the time. I admit that sometimes when writing a section of an application, I don't fully understand the problem I'm solving. It's only after seeing the solution that we have enough information to know how it should be solved.

However, we don't always go back and correct our original implementation. When we don't, we accrue technical debt. This type of debt usually has a very high interest rate, and will become painful in the near future as you'll find trouble implementing features that use this sub-optimal implementation. Therefore type of debt is best payed off as soon as possible, preferably even before committing the code (or pushing to master in a DVCS). If left unchecked, this can quickly put an entire application under water.

Hidden Debt
Financially speaking, hidden debt is when you don't fully understand the impact of a regular debt, or don't even realize that a debt isincurredat all. This can happen if you don't read the line item in your loan that says if you don't pay it off in 12 months, the interest rate will jump to 125% (yes, these loans do exist). When programming, we make decisions that we think are right at the time, but turn out down the road to be bad for the application. Don't fool yourself into thinking that these are just normal work-flows for development, it is technical debt as well. And it's also the most dangerous form of technical debt, because you're likely not even aware that it exists.

Putting It Together
As you have probably already surmised, each form of debt is applicable in different situations. My standpoint is that when working with a team, each form has differentacceptablelevels, and need to be approved by the appropriate part of the team. Payday Loans have no place in a production codebase. If you need to do it because you communicated the concerns to the business, and still HAVE to in order to make a deadline, then the entire team should be part of that decision, and should ensure that it'srepaidas soon as possible to prevent it from taking down the entire project.

Mortgages on the other hand are quite useful in a team setting. But the entire team should make the decision whether or not to take on the debt. By expanding the decision to multiple people, the edge cases for the additional functionality can be more clearly seen, and it can be dealt with appropriately. Credit Card debt I think is a judgement call that the individual developer needs to make. Are you going to push the sub-optimal code to the repository? If not, then use your judgement. If so, let your team know so that they are aware of the debt.

Hidden debt is harder to talk about. Because you don't know it exists, you can't by definition decide if you want to accrue it or not. Instead, when you find the debt, you have to categorize it as one of the other three types (as best you see fit) and then decide whether or not to repay it now, or accrue it for later. But the most important thing is to acknowledge the debt, and don't just ignore it.

How to Identify Hidden Debt
I think that it's worth noting a few methods to identify locations of hidden debt in your application. If we look, there are basically three forms of hidden debt:

Architecture Debt - This is basically when the debt is incurred in the application's layout and the way classes and objects interact with each other. They tend to be more pervasive and harder to repay than other forms of debt.
Design Debt - This is when you structure the component correctly, but instead muddy the API with poorly named methods. This tends to be pervasive, but reasonably easy to repay as long as you catch it early. If not, you can wind up touching a lot of code to fix one simple naming convention.
Implementation Debt - This is where you accumulate debt in an implementation of a method. This tends to be very localized, and can usually be repaid very easily.
Implementation debt is actually one of the easiest ones to identify. Look at a method. Does it look to be doing anything wonky? No? Then you're likely debt free there. Automated testing tools such as PHPMD can really help with identifying and preventing certain forms of implementation debt, such as having a method that's way too long (lines of code), does way too much (NPath Complexity) or is using poor practices (exit(), goto, eval(), etc).

Design debt is a little bit harder to identify, in that you need to think about the object in its use-context. When you read a method's name, does the method answer the question "What does this method do on this object"? If not, it probably needs a better name. An example of a poor name is SplObjectStorage::getHash()... What does that do? I can't tell without referencing the documentation. An example of a good name is: SplObjectStorage::removeAllExcept... I can almost read it like a sentence: "This method removesAllExcept (param) from an SplObjectStorage instance"...

Architecture debt is a lot harder to identify. Really, the best tip that I can give to identify it is that if you find yourself trying to solve a problem that's tricky with respect object relationships, your architecture is probably flawed. A good example is if you want to prevent child objects from making a protected methodpubliclyvisible. This is non-obvious, and isn't something you'd normally do. To me this triggers an instinct to look at why I need to solve this problem in the first place. Usually, it's because I'm trying to use inheritance to force one class tree to do the job of two or more trees. Really, it's more of a feeling and a judgement call that can only be made with experience.

But Technical Debt Doesn't Exist With Agile!
While researching for this post, I came across a rather interesting post claiming that the concept of technical debt does not exist when using an agile methodology. The post does make some very good points on the surface. But I think his conclusion is actually flawed. He makes a very good case that Payday Loans don't actually exist in this context, since from the view point of the customer, the feature is working, therefore it is just trading off current velocity for future velocity. That sound very much like the very definition of technical debt to me, the ability to amortize short-term gain over the long term. You can call a pot a chicken, but in the end it's still a pot...

One point that I do agree with is that Mortgage debt isn't a negative in the realm of agile. If you don't need the feature now, don't write the code for it. I think it's important to recognize that it's still debt, as it can be non-obvious in the long term why something appears half complete.

However, one point that I think he gets wrong is the concept of Credit Card debt. He equates it to poor code that should be corrected by review, training and pair programming. While all three steps can help, they won't always be able to identify the true impact of the decision at that point. Sometimes it won't become clear how to properly solve a problem for a significant amount of time after it was written, even among a team. That doesn't mean that the programmer was deficient for writing that code. It just means the requirements weren't clear enough at that point in time. This happens all the time, to even the best programmers. Identify it as debt, and move on until the solution becomes clear. It's better to spend your time doing productive work, then mucking around in something you don't understand. But don't forget about it, revisit it form time to time.
Technical debt in the commercial world is an idea that code owned by the shareholders of a company is accumulating debt created by the coders. The coders feel they have been forced to get the share holders into debt because the executive forced features out too fast.
I think that this is applicable only to a subset of the reasons for incurring technical debt. We incur technical debt because we realize that sometimes there is a difference between Good and Good Enough. And we don't ignore that distinction, but embrace it. Once we get to the point that we can accept that as a good thing, and not as a negative or limitation, then we can truly be professionals. Technical debt is not about blame, but aboutadmittingthat we are not perfect. I don't know about you, but I know I have written some really wonky code in my life... Let's accept that, and embrace technical debt as a powerful tool to embrace our imperfection and solve problems better in the long run...
Now What?
Well, at this point, we've decided that we want to accrue technical debt to help us develop an application. This is a great thing! But we must be careful. Once you decide to accrue technical debt, you must track it. And more importantly, you must dedicate time to repay the debt along with ongoing development. One method for tracking the debt is to include comments right in the code indicating what and where the debt is. However, that requires us to look at the code to determine where the debt is. How often do we see a TODO comment, andsummarilyignore it. I'll be honest, I do it all the time.

Instead, I'd suggest treating technical debt as an application feature or requirement. Are you using SCRUM (or a similar style agile system)? If so, put in user stories for your technical debt (as in, one story per item). That way, it'll always be in front of you during planning sessions, and it's in a place where nobody can hide from it. If you're using more of a waterfall approach, consider adding technical debt to your issue tracker as actual bugs. And prioritize them appropriately.

You don't have to make the debt repayments a high priority unless the specific debt is going to pose a problem for a feature you're working on now. Just make sure you keep it in mind when planning new features...

As I said before, don't be silent. Don't ignore debt, or it will cripple you. Accept it gladly, because if you use it properly it will make you much more efficient and produce higher quality software faster. But also remember to be prudent in how you use it. Don't accrue debt just because you can, or because you want to impress your boss with how fast you can get something done. Be smart with how you use it, and you can do amazing things in surprisingly little time...

I'll leave you with a final thought:
It's hard enough to find an error in your code when you're looking for it; it's even harder when you've assumed your code is error-free. - Steve McConnell
What do you think? Feel free to leave a comment, follow up on twitter, or post a reply to your blog!