technical debt

This is a transcript of the episode #224 of Software Engineering Radio podcast, available at http://www.se-radio.net/2015/04/episode-224-sven-johann-and-eberhard-wolff-on-technical-debt/. The audio recording is licensed under the terms of Creative Commons 2.5 license, which allows redistribution of the copyrighted material. All rights belong to the original copyright owner.

This is not an official transcript of the episode, and therefore it may contain minor gaps and some inaccuracies.
-----------------------------------------------------------------------------------------------------------------------

00:00
<Music>
This is Software Engineering Radio, the podcast for professional developers on www.se-radio.net.
SE Radio bring you relevant and detailed discussion of software engineering topics at least once a month. SE radio was brought you by <…> software magazine, online at computer.net.org/software(?).
So, we have come to another episode of Software Engineering Radio. This time around we are going to do a special issue, so that's gonna be your classic interview, sitting here with Sven <…>. We both collaborated on an article on technical debt, so we will do an episode for an IC radio about technical debt. It's not going to be an interview, rather we will talk about thoughts we will sort of share, that's the whole idea. The article we did, we will have linked to that article in the show notes, so you will come to take a look at it. So before we start, Sven, why don't you say a few words us about yourself?

01:17 Sven: My name is Sven <…>, and I work as a software developer for <…>, and I'm also involved with TRIFORK's Goto Conferences, and I'm exited to propose??? the Microservices MCDM meet up.

E: So, sounds like a good load of fun. So, my name is Eberhard, I'm a freelancer, <...>, I am a speaker mostly at German conferences, and I've just authored a book about continuous delivery (also in German), I am also involved in GOTO conferences and several other conferences. So, let's jump into this subject. Before we talked about technical debt. Technical debt is obviously connected to the quality of software, so before we dive into the technical debt discussion, we should talk about quality. And if you talk about quality, there are actually two different types of quality. The warm(?02:19) type is external quality, so that's the quality that can be perceived by a user or a customer. That's might be the performance, might be the other things like a security, scalability or the software stability, and so on and so on. And it can be measured or experienced by users. Because, it's a feature of the products, so to say, and it's actually something that should be managed by the product owner or some business people, because they are interested in the quality or how the software can be perceived and used by the users, so it is just like buying a car that have certain qualities. It's just another feature of the product. The more complex part of the software development is that there is also internal quality. Internal quality can only be perceived by developers or technical people. It's anything that makes extending and maintaining the code easier or harder, so there may be things like they are tested, or they are missing,  if there are more tests and code is easier to change, and may be able? architectural styles, or architectural problems. It's also about coding issues, whether the code is too complex or too easy. And the hard thing about software development is that  this part of the quality cannot really be perceived by anyone, except for technical people; and that means that appraising?? internal qualities is actually quite hard, because if you are not a technical guy, then it's hard to see what this quality really is, and how it influences the development process.
That is something that is also important for technical debt. So, Sven, do you want to talk about that part?

04:11
S: Basically, you can say technical debt is a metaphor to describe "not quite right code". The debt is a comparison to financial debt, and we use it to communicate this inner quality and risk of a codebase to non-technical people. The point is, why should we software developer talk always about quality code, and good code, but non-technical people usually don't see the benefits of bad code. For them, it's just "the software works, and fine! Why should I invest in quality?" But the technical debt metaphor helps us to communicate that, if we want to build something on top of not-quite-right code, it's getting expensive to do something on this codebase later on, so it takes longer to implement a new feature on a not-so-good codebase. Also, an internal quality become sooner or later an external quality - that's also something we have to communicate. For instance, if we had a bad codebase, and then we get more and more bugs, we slow down, and that will eventually bubble up to all kinds of stakeholders of our project. So, technical debt is actually not a developer problem - it's company-wide problem. And if you have too much technical debt, in extreme case the whole engineering department stands still. I think, a lot of companies experienced that already, with 30-years old Q-board systems, where they basically were not able to change anything anymore.

06:06
E: So you could argue that technical debt is actually one of key points to successfully - commercially successfully - develop software. The term was actually coined by Ward Cunningham in 1992. What he said is: "Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite… The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise". This clearly says that the technical debt metaphor is closely related to financial debt, and it's about shipping something quickly by going into debt. And then you need to repay the debt by increasing the quality later on, and if you don't do that, then you will have to pay interest rates, because your proactivity goes down, and you are slowed down in the development. As Sven have just said, this is a good metaphor to talk to management, because they should be familiar with the financial terms, and it is easier to talk to them in these terms and tell them: "This is just like getting a loan in the bank: you have some benefits for some time, but then, at one point, you need to repay it. And you need to repay it plus rate." That's how the term was coined. But there is also a different definition, right, Sven?

07:47
S: Exactly, exactly. The research community rather use the McConnell’s definition. He says: Technical debt, that is a design or construction approach, that is experienced in the short term, but that creates a technical context in which the same work will cost more to do later than it would cost to do now, including increased cost over time. I think that is the definition which most of us know as "Technical Debt": that we do something quick and dirty, and we gain something on the short term, but we know it will bite us in the long term.

08:30
E: So, there are two common.  So after we explain the metaphor, i think the most important part is how do you actually deal with technical debt. As I said, it's actually one of the keys to the successful business around software development. The questions is how do you deal with it. The first question probably is it an bad thing at all. So if you think about it, going into debt is often not such a bad thing. If you buy a house, for example, it may actually save you some money afterwards, because you are not going to pay rent for an apartment anymore.  That kind of investment may be a good thing. That's also why there are loans in the industry, because they give you some money to invest in better machines, so whatever... and get better proactivity produce better payoff. It's not necessary something bad, and in some cases - again, if you look at the metaphor - it might be bad: so if you're just going into debt to buy some luxury items, or presents for Christmas, then it is not money that's invested - it's just money that's spent, and it will be gone afterwards. Concerning the metaphor, and talking about the software development, it is not always senseful to pay back the debt. Why is that? The reason is because, comparing to your normal debt, the interest rate that you are paying only occurs if you change the code. As we said, the internal quality is around how changeable the code is; if you don't change the code, then the internal quality and the technical debt doesn't really mater at all. Paying back technical debt in part of the code that are not changing doesn't really make any sense at all.  The other thing is that sort of the Miner-parts of the software development game: the person who will pay back the debt is not necessary the one who incur? the debt in the first place. If there is some project that creates some software, and there is a different team that is going to do the maintenance afterwards, then the team that does the implementation will not pay back the interest rate and also not the technical debt,  but that's going to be the maintenance team. In that case, it might be in the best interests of the implementation team to pile up technical debt, because they will just go faster, they will never have the downside of the whole team, because that's gonna be the maintenance team. That's one of the problems that you may have wrong (with) technical debt, that the person who causes the debt will not be the one who will later be responsible for the code.

11:26
S: Also, awhile?? we go ahead in discussion what's coming here?? about technical debt, and he said technical debt is actually a strategy. Why it is a strategy? Because we can quickly breach ?(reach?) a business goal going into that, because, for an instance, it's much more important to bring something on a market very fast and that more important than, for instance, having perfect code and being late, but we can throw something on a market and see; and may be we are the first ones or we see if it is the first one at all. I think interesting is that, for instance, Eric Ries they offer of the lead’s data. He describes it in his book and he says that, when he work at startups, he was always so happy that he wrote this perfect code, but in the end nobody used it, so the functionality he created with perfect goals was totally unused; so it's better to build something quickly, and bring it in front of the user, and just see is it actually useful for anybody. And if it's useful for anybody, - okay, than we can pay back the technical debt. If we create perfect code for functionality we don't know if it's useful or net, it's basically the waste of time to do that. Henrik Kniberg from Spotify wrote a year ago a blog post describing were he described the same, where he said: "Basically, we see perfect code as a waste if we don't know that a functionality it creates is really-really-really useful”, so what they do is: they come up with the functionality extremely quickly. It's not in very good shape, and they bring it in front of the user. If  they see the user likes it, and they want to build on top of that functionality, then they refactor it, and they make it nice. We see, with easiest strategies over and over again, I think, the famous cases of Amazon and Twitter... I think, with Twitter somehow it feels they are rewriting the system all the time, because initially it was a Ruby-on-Rails application, and now it's a messaging system. Amazon was also a very different system in the beginning as it's now. Yeah, I think it is a strategy.

14:05
E: Right, if you look at the Amazon, <??name> part of books actually talks about his??, The City of Amazon, where he said at the beginning it was just gigantic monolithic application that was really hard to maintain when he joined the company, and they did the complete rewrite, and in the process they came up with all the nice infrastructure elements that are now known as the Amazon cloud. The point to make here is that if Amazon had designed the software with high-quality in mind in the first place, and with the great scalability, they would have probably not survived, because they couldn't bring out the features as quickly as they would need it to. That's basically where the software was completely rewritten, because there was so much technical debt, and at the end the companies were enormously successful.

14:59
S: I think for Twitter it was similar, because in the beginning they had no clue how many users do they had, so it's a difference if I have a thousand users, or a million, or a hundred million of users. You have to start somewhere.
E: By the way, that is also something that I would note if I were made talk to software architects. I give them some excess, how is they often come up with suggestions that have scalability in mind,  and I think that scalability is they main concern, but in fact they should probably focus on time-to-market more, because otherwise they won't see the point when they will need the scalability, because then the company would already be bankrupt or the business case would have be gone. So, as you see, it's not necessary always a bad thing, still you need to deal with technical debt somehow. And one of the great ideas that I came across at one point was by Eric Evans. He is famous for writing this book about Domain-Driven Design, and there is this part of domain design that basically everyone knows - about ubiquitous in language about repositories or this kind of things, and there is also a different part of this book that not too many people seem to have read, and that is about Strategic Design and design on a more coarse-grained level. What he says is, first of all, you cannot have the same quality of code throughout the whole system, and if you think about it, that's actually quite obvious, because you will have good and bad developers in your team, and even if you had a very-very good team, there were still be better and worse developers, so you won't have the same quality  throughout the system, cause there are just the different people in the team. What can you do? While you can just leave it to chance which parts have the better quality, you can make it a conscious decision. One thing to deal with technical debt is to ask the question: "Which part of the system are really important, concerning changeability? You might get that information from the historic data, like where were the last few changes, or you can think about it more from a business perspective: so, which parts, if we can change them quickly, will give us a competitive edge, like, for example, the way that you do your shipping is very important, and in that case the part that does the shipping should be of high internal quality. Eric Evans, in his book, has quite a few patterns that talk about it, so best of what he says is that you can have a bound of context, in which certain domain is more or less valued, and you can have an anticorruption level between those bounded contexts. If apply that to the problem of internal quality, what can do, you can create  a part of the system - let's say, this is the shipping system again - you have a really nice domain model there, where the highly-sophisticated and the code of high quality is, and you have a different part  of the system, for example, the part that deals with the customer, and there are just tenants?? of where is has low quality, and a pretty awful domain model. To make sure that this awful domain model doesn't leak into your invaluable shipping system, you put an anti-corruption layer that separates those two different models from one another and translates between them. Using that approach, - I think I would put it into use as a metaphor, as "cells" , so like an organism which is built of cells, you build your system out of several different-bounded contexts, which have membranes just like the cells, which is the anticorruption layer. They makes sure that the external problems don't leak into the cell, into the bounded context. And they are clearly separated, that you can have sort of cells or bounded contexts which is of really high quality, and if you have quality problem, they don't release to spread throughout the code. It is the nice way of strategically developing the quality and the technical debt. What you do is, you decide which parts are important, and you do care about those, and you have your best developers working on it, you monitor if the quality close the ends so on, and there are these other parts, where you might even use standard software, the software which you bought or you can just stick to your legacy system or whatever. So it think that's an interesting way of strategical development of the quality of a rather complex system.

20:02
S:One questions is, of course, why there is technical debt at all? Why do we have it? Can't we just create perfect software from the beginning? I think, as developers... almost ever developer wants to deliver great work. I think nobody  has the attitude to deliver low-quality work. We all want to do good stuff. Why do we deliver low-quality work if we want to deliver high-quality work? There are several reasons for that, and the most obvious reason for most of us developers is time pressure. So we want to do something really god, but we have no time. We have to ship, we have a deadline, so in order to be faster we compromise on quality. We just script testing, we aren't automating tests, we don't think deeply about good design, and that leads most of the time to not-so-good quality. Another point is that we are missing know-hows. If we use a new technology the first time and we have only a little understanding about that technology, we will inevitably use it wrongly, even if we have the best intentions. Also if we don't know the business domain very well, that also leads to strange designs. There is also something like "software decay", so even if we have created software, if it is like EGB2.0 was may good like 15 years ago, but know even that perfect system is a bit outdated. I think, there was a very interesting study - Surrey from the University of Tokyo. They ask people in Finland where their technical debt comes from. It was really interesting, because they didn't say "time-pressure"! They also didn't say "We didn't know the technology". They said: "We either didn't understand the requirements, basically we didn't understand the business domain, so technical debt comes from not well understood requirements"; but also what they said that if we had a bad architecture, or architecture which is not communicated, that was for them the main reason for the technical debt.

22:40
E: That's actually quite interesting, because at one point I also thought that a technical debt is wrong???, but the architecture may be the major source of technical debt. Then, at one point, I asked myself: what is it that makes soft actually hard to change? And the answer that I came up with is tests. So, if I have had the choice between the system that has a bad architecture but a lot of tests, or a system that has a great architecture, but no tests at all, the system that I would want to change will the one with the bad architecture but the tests. That is again is one of these points where I'm actually not too sure how to define technical debt and where I just surround(?). Well, you could qualify an architecture with just sort of traditional part here.

23:35
S: I don't know. I've recently heard that Philippe Kruchten, the inventor of the  Rational Unified Process talking about that, and he is known for a lot of architecture works, he said: even if you have lots of test cases, but bad architecture, you cannot just assume you can refactor better quality into the system; because, if the architecture is wrong, - if you, for instance, chose the wrong stire(?style?24:05), refactoring doesn't help you anymore. You really have to re-abit(??) the system anew. I don't know, actually, it's a good questions: what's worse - bad architecture without test cases or okey-architecture with test cases?

E: That's one of the things that you can debate, I guess, for quite a long time. Concerning the sources of technical debt, there is also NATO model, which Martin Fowler came up with. He is obviously known for quite a few things in our industry. He said he would distinguish a technical debt into four quadrants. And the quadrants are separated by whether the technical debt was Reckless or Prudent, so best kinds the questions will stay enough time to even think about technical debt; was it just Reckless, when people where on a lot of pressure, or was it prudent. Still in that case, if you have enough time, you can still get enough technical debt, as we will see in a minute. And whether it was Deliberate or Inadvertent, we will have technical debt here, but we just do it still or did it just sort of happen, so we don't have any time for design, so we'd just do it anyway? And then you end up with a system that is badly designed, but you knew that would end up with this situation, so it was a conscious decision - it was made because it was just not enough time? So would debt will be a reckless or deliberate.
Prudent and Deliberate would be "We must ship now, until there are consequences", so even though there was enough time for dealing with technical debt, we still did it, and it was a deliberate decision, so we actually said: okay, we will end up with some technical debt here, but we still do. Inadvertent -and-Reckless would be "What's the layering?" ?? and it's obviously something "You don't have enough time to actually think about it" and think it's true; and it was not a deliberate decision because you didn't even know that it was an issue, because you didn't know about what is the layering, so you ended up with non-layered system, and a lot of chaos, because you didn't think about it. And a Prudent-and-Inadvertent would be a that you end up with your system, and then in retrospective it becomes clear that the system should have been done differently; a quick fixture that you came up with, at this point it was okay, and during the implementation you came up with a completely different idea about the system, and that's better idea, and then in retrospective it became clear that you have a lot of technical debt. And it was just that it didn't occur to you that you should have done it differently. So there was a lot of times that it is quite possibly a prudent decision, but it was also inadvertent, because all the facts about system and how the architecture would behave wasn't clear in the beginning. What I think is very interesting about those things is what we said in the beginning, when we were talking about the definition of technical debt - it was around going into technical debt to ship features faster, so it is something that is a conscious decision, so we are going to debt, and that for we will ship features faster for some time, and then later on we will pay back the debt. What I think is interesting about the Martin Fowler's quadrants model, if you think about it more closely, that is actually always the case. In some cases, it's just that it wasn't clear to you, that you should really do things differently, because things just are clear in retrospect, when you think about it again. And if you think about how the whole thing that you did the architecture for worked out, and also I think a different take on technical debt, because he says it's reckless; so it's not about saying "okay, we don't really have time", it's just there are so much pressure, so you just go ahead, and that the technical debt piles up, but it's out of conscious decision, or it's just something just happens, because you are under so much time pressure. I think the interesting part to take away here is that there are different sources of technical debt, and the simple model that says "okay, we compromise on quality to go faster" - it's actually not always what happens. I think that splendorous?? set has an excellent point here, so if you start with a system which use the system like the EGB2.0, obviously it becomes a technical debt at that point when there is a new version of EGB2.0 or your favorite library, or whatever, because then there are easier ways to do things, and the way that you implemented your things are technical debt. There was not even the slightest chance to avoid that technical debt, it's just because it's technology, innovation and so on. This is a sign, in my opinion, that this metaphor breaks.

And next question is, of course, is whether it's realistic to have a debt-free system. I think that's something you wanna talk about. Right, Sven?

30:04
S: Exactly. Is it possible to have a technical-debt-free system? I mean, I read quite often something about "No more technical debt!" and "How to be debt-free in 10 steps" (or something what you have to do to be debt-free). I think, to have a debt-free system - that's impossible. We should just accept that there will always be technical debt. Even if you have a debt-free system, how have you archived it? You probably had to invest a lot of time and money, and that is not necessary tied to the success of the project.

E: The key point to take away here is, if you are thinking about it, is the  original?? implementation, that we discussed for MS or Twitter, they were hugely successful, but there were quite a lot of technical debt. I think, one of the interesting parts here is that technical debt is not tied to the commercial success of project at all, so you can have enormously successful project, or a business, that is based on software which is full of problems. Then you can do rewrite, so extreme programming came up with the idea that the quality tied to the maximum, and just have no compromises about technical debt - that might be a bad idea, because then you invest a lot of resources, money and effort into maintaining high quality, even thought it's not necessary, and might not even influence the commercial success at all, because it's just something that the user doesn't even see at all. So, Technical debt might not be the best metaphor, and the question still is why to be even caring about technical debt. Well, why do we care, because in the long run it might influence productivity and the technical debt is just something that we use to improve communication. Maybe it's not the best metaphor, because if you say that a debt-free system is impossible, and you say that in a lot of cases there aren't even a conscious decision about the debt, and in some cases because of new technologies technical debt just happens and you cannot do anything about it, then the metaphor is sort of broken to an extent that you might need to look for something else. And that is something that Felix Mueller? tried to work on, during his master thesis. The basic idea is, if we cannot use "technical debt" because there is always technical debt, because in many cases there are no conscious decision, why don't we try to use a different metaphor? The metaphor that we came up with is "Quality improvements". The idea is: instead of talking about debt...
S: Was it "Quality improvements" or "Quality investments"?
E: Improvements, basically, but we also use the metaphor from the financial world, so metaphor that we are using is "Quality investments". The idea is, if there is always technical debt, and if there is always something that can be improved in terms of internal quality, why don't we think about that as an investment, so we INVEST in quality, so we have someone to work on something for a few days - let's say, we improve the testing of our system, and that takes like five days. And we will have certain pay-off, so we will have... we will probably save one or two days per sprint, and then we'll have a payoff. That's just like any other investment. The idea is that there is a cost reduction, and the cost reduction is, basically, the better productivity, minus the effort that you invested in the quality improvement. And the last thing about it is that there is a quality model that thinks along the same lines, I would say - that's the SQALE model - it's S-Q-A-L-E - and they have remediation cost, which the effort to eliminate quality problem, and non-remediation cost, which the effort that you need if the quality problem is not eliminated.  If you have a bad code coverage, the remediation cost would be the effort to improve the code coverage of your tests, and the non-remediation cost would be the lower productivity because you'll need to invest more time in manually fixing bugs and all these kinds of things. This is, I think, quite an interesting thing. At the core, I believe it's around a mindset. The mind set should be: if I invest in something (if I improve the test coverage, if I improve my continuous delivery pipeline, if I invest in code quality), it should not be gold plating - it should be about things, actions, that  keep the proactivity high, and they pay off. If you think about quality in that way, I think that's a good way to handle technical debt, to handle your internal quality, and also to communicate it to the decision makers, the managers, because they are used to investments in the way investments works. And, one of the criticisms to that model or general criticisms, is around "The developers should always keep the quality high!” I believe that there are still issues that need to be resolved at a becow???, so that cannot just be done by developer by himself, but rather they need to be with the product development and so on. For example, if you want to have a continuous delivery pipeline, because you want to go quick into production, then that's gonna be an enormous effort, and that is not something that a developer can do by himself. To convince management to invest into it, a metaphor like "Quality investment" seems okay: by this continuous delivery pipeline, we will have much better quality, and we will have less bugs and high proactivity and so on. I think that's is a discussion that you can have with management and then end up with a bit of solution, basically.

Okay. Still the question how you can identify a technical debt, and that is where you have some thoughts, right?
37:26
S: Yeah, you've basically stolen some thoughts from other people. There are some obvious identifications, like if the code is full of TODOs and FIXMEs, that shows that the people went fast to deliver, for an  instance, the functionality, but why doing that, they already saw a lot of cases where they, with these TODOs, were saying: "Hey, we need to externalize this configuration!" or "FIXME" because "This works only for 95% of the cases, and we really want it to have this feature, we'll need to take care of that". I think, this is easy-to-fix technical debt. There are some serious things, for an instance, if you want to reassort??(38:22) a code change, and then a developer says "O! I cannot change this piece of code, because this is cat's code!", then you are in a big trouble, because it means that you decided to go fast and only one developer had everything (the whole model in his head). It's not shared, it's a knowledge island, and if that developer is hit by bus, then nobody is ever able to change anything in the codebase. If somebody says "This code can only be changed by person XYZ", that this is a clear indication of a big technical debt. Also, if you hear it the whole way, quite often: "Yeah, we have similar functionality here and there - let's just copy and paste the code, changing in some places" - you can do that maybe a couple of times, but if you do that too often, then your codebase becomes totally unmaintainable by "Copy-and-Paste strategy" thing. Another thing is also if a developer says "Ooh, I don't really like to touch this, because I'm not sure. If I touch this, then a lot of things will probably break" - so this means, there is a really bad design and you cannot make local changes to local requirements, because everything is kind of tied together and spaghetti-wise, and  then you are at risk that you go very slow, because every new requirement you implement, or every code you fix, basically, introduces new bugs, and your only way to avoid it is intensive testing.

40:07
Also, an indication is, in Scrum teams for instance - they have velocity - if the velocity of the team just goes down and down and down and down, and the team is basically stable. Then you should, of course, ask yourself, why is your velocity going down. And that's mostly technical debt. And a point we had before - software ages. Old libraries, old GD cases, old everything, - even if you had perfectly designed system, ten years later it's not that perfect and you have to update or upgrade libraries, or even exchange libraries, to do new stuff which is required in, let's say, 2014 to be successful. It's also bad if you hear something like "Ooh, we can't fix that important bug. It's only one line, but it takes a week or two to bring the system into production". Actually, what you want is: you fix a bug and, ten minutes later, or maybe an hour later, you can put a new version into production. But if you have a cycle, like, two or three weeks to fix a little piece of code and bring it into production, then you also have a problem.
41:29
Not so important, but interesting, is, if you have very-very slow tests, that's obviously costs time. And if you have a policy that says "you have to run all the tests before you check in" (I think, that policy every project should have). it makes a difference if you wait a minute before you check in, or if you have to wait ten minutes, or twenty minutes before you're able to check in - there are couple of other ones, maybe you want to talk about it, like SonarCube or Structure 101...

E: If you look at the quality of your code, or if you want to look at the quality of your code,  so make sure you have a personalized tool that does a lot of different things: utils that start a code analysis, with different tools that also do code coverage for tests, and a lot of other things. The nice thing about it is that it integrates a lot of tools and gives you a very good idea what code quality of your code is. And it does gives you a historic data, so you can see whether you are actually improving or whether things are getting worse. And there also tools like Structure 101 that I kind of like, also SonarGraph that allow to enlist a hypo trophic code. Just put in your code - you compiled code, often - and then you take a look at the architecture and see where the architecture is nice, whether there are a lot of dependencies or cyclic dependencies, even whether the architecture is too complex, so you can acknowledge than. Those are tools that you are in use there. Again, I think that the most important thing is to have code that is easily testable, and if you want to identify technical debt, what you need to think about is what has the biggest impact on maintainability, what has the biggest effect on changeability, how can I improve things there? That's not necessarily something that is tied to the code, it can also be tied to the tests, for example, low test coverage, or the code is hard to change because you cannot trust your changes (they might have broken something and won't note this until your production), or, as we said, if you have a slow deployment or slow tests, there are just fewer changes that you can do to your codebase, because putting those changes to production takes such a long time.

We've spoken about identifying technical debt, so now the question is what makes can we do about the technical debt.

That a very interesting thing. One of the ways you can deal with technical debt is to create a buffer-task pre-release. You could say "Okay, let's allocate episode of the time to the team, and have the team work on technical things, and this thing will improve things". And you can even spend more of your budget on technical debt,  so you could have technical releases that just improve the codebase; that means that the effort invested in handling technical debt is not evenly distributes, like in those buffer tasks that have tempus of the sprint. It is a whole sprint every, I don't know, two or three sprints, that is used on the codebase. You may have a technical backlog, with stories for the things that may be improved; one other way of doing it is to say "Well, if we have a task or a user story, for a business feature, then we can also have some effort invested in handling technical debt". Basically, the idea is: here is on your story "We want to change a registration process", and what we do is that we change the registration process, then we improve the software quality to make it easier to implement that registration process. And that way we invest the part of time in improving the quality in those areas of the code where the changes are actually made, and it's also factored into the use cases, so it's something that can be decided about by management. They can say "Okay, i don't wanna do this story, because it's so awfully expensive, because the quality is so low" and then we can have management do those decisions. However, in the end of the day, quality and handling quality should really be a  business decision, because it's about prioritizing quality over features. That is, as we discussed, quite an important thing. If you improve quality, that will pay back in a long term. However, if you really need this feature done, otherwise your business case is gone or there are other severe business consequences, then the quality doesn't really matter. The question whether you should invest in quality or not should really be a business decision. Another problem is if you have just allocate  episode of the time of the team for handling quality, or if you have allocated certain releases for handling quality, then it's not a business decision anymore - and it really should be the one.

I think, the hard thing about handling technical debt is to enable the business to decide which part should have higher quality and where to invest effort. If you had used an approach like "The team will know how improve the quality best and where to invest best", that's is clearly  not the case: then the TEAM will decide. They might not be able to do so, or they might not be able to reach the best decisions, because they would not have the full picture. And that is why, I think, getting the business on the same page, that's one of the hardest parts. Communication with management is also the core of the whole thing, of the metaphor of technical debt - that is why it was introduced in the first place, and to some extent it is also about trust. If the developers know how to handle quality and how to keep it up, then they can have them decide about where quality should be improved. Otherwise, you would need some kind of decision <...0:48:10..> to invest on quality and that might be just very consuming and hard. So, I think it's about trust, but it's also about so-said "investing", where you can get a payoff and that is something that really the business should know about, and should think about.

48:28
S: Yeah, I think trust is really important point, because quite often yet the case is developers, complaining a lot about bad quality and bad frameworks, and they say: "ooh, we need to improve this, and we need to change this framework, and to do this and that" - and they remain (or we remain) unheard by the management. They just say: "I know, it's just not so important". But, for instance, if you have a very good developer, who is proved to understand the business, and the business people trust this guy or this group of people - if they say "I think we should really tackle this problem, because we are going nowhere with that codebase. We should clean it up and only then it makes sense to go on" - that the management people, if they trust this developers, they say - "Ooh! If Fred, and Mike, whoever, if they say this is really important, and we really have to do that - and they in general understand the business very well - so maybe we should listen to them and let them decide what to do"

49:44
E: To some extent, what you saying is right, but on the other hand, the business people are the only ones who can actually say about whether investing in quality right now makes any sense at all, because we just need to ship certain features, and also, as we discussed, technical debt depends a lot on which parts of the system we are changing in that feature, and that's something that developers can only know to some extend, so they only know what's going to happen in the next few releases, but the long-term strategic vision is something that the business should come up with. That is why, I believe, it is not just about handling over the responsibility for the quality for the developers, it has to be an effort by developers plus business, because they both have certain pieces of the puzzle, and they need to put them together to come to conclusion where how and when there should be investing quality.

S: Yes, but what I meant is there are developers who are domain experts, and they kind of understands also the business strategy and where the system is going, and if they the business people, they trust this developers, and they say "We should really invest in quality here" - mostly, they are heard by the business people. But, if you don't understand the business, then it's going to be really difficult to build up that trust.

E: Right. If you have had such developers, and such a team, then you are very, very lucky, but that also raises the bar for developers to a level where I'm not sure you can have a lot of people who can actually do job; because then you are not just looking at the people who are really good technical, but also know a lot about the business. It's already hard to be a good technical guy - to be a good developer and to be good in developing code and doing architecture - but if you are then also supposed to know a lot about the business, that's gonna be rarely??<52:11>  hard. That's why I think in some cases you need to have a business and technical people work together on this. That's why it so hard.

S: Yeah, we have, I think, one important point left. There is a question that Frank Buschmann raised a while ago. Frank Buschmann  is widely known for the article "Pattern-oriented software architecture" and he asked: "To pay or not to pay technical debt?" I found this article was quite interesting. and unfortunately it's not freely available, so you have to buy it from <....>, but he said "Okay, we always have technical debt; it's unavoidable; how can we do with it in terms of 'Do we pay back or not"?"

 He gave 3 answers to that. He said "Okay. Point 1: The Debt repayment. We have a really bad piece of code or component of our system, and we decide to completely refactor it or to, maybe, even completely replace and throw away the code, the framework, the platform, whatever, which gives us headache, and pile it anew. Or refactor it to a stable and good design. You should only do it if the code is really bad and you know you'll have to build a new functionality on top of that in the future very often.".

The second point he proposes is the debt conversion, so you have a component or a part of the system which has very high technical debt, but replacing it is just not a solution you can do. For an instance, you have a 30-year-old legacy application. You cannot just throw it away, because it's just too expensive, too risky. What you can do is the debt conversion: you try to transform the system to a good, but not perfect, solution. And this new good, but not perfect, solution, has a lower "interest rate". It's still not perfect, it's still painful to build new stuff on top of it, but it's much better than the old system. We cannot afford to create a perfect solution, because it's too expensive and too risky to do that - to completely repay it.

And the last one (I hear that quite often and I think it's a valid point), that we just accept technical debt at some point. We just pay the interest. We know the code is not very good, but we live with it. It's not very good, but we also do not lose so much time and the cost of refactoring the not-so-good codebase to a good one. It's more expensive that just working with not-quite-right code. It's something that we should always have in mind: should we really make it good, or does that costs more than living with that?

I think that was really a good contribution.
55:46
E: Okay. I think, that's all about it, right?
S: Yeah. Thanks a lot for taking the time and talking with me about technical debt. I hope you have enjoyed the show, and if you have any questions, I would be happy if you could leave some comments on our page, so we can interact with. Thanks a lot for listening!

<Music>