Hacker News new | ask | show | jobs
by moxious 2888 days ago
I've been the CTO at two tech startups. Here's how I think about it.

"Debt" is a good metaphor because by accumulating it, you get advantages right now in speed and ability to focus on other things, in exchange for having to pay it off later. You can either pay it off by stopping what you're doing (uncommon) and fixing stuff, or you can do lifetime installment payments of more complexity, bugs, etc.

Tech debt is just one aspect a CTO has to manage. Time to market is pretty damn important. Features are important. Many other things too. This is not to say that tech debt should be ignored, but elegance of the system is not the paramount concern. Just as startups borrow money and take financing to move quickly, so too a responsible amount of tech debt can be a good thing.

In engineering we tend to over-emphasize how important the technical qualities of the system are because that's what we see every day. There are a great many things that are also important, and various business scenarios where they become MORE important than tech debt. But if you're in the stage of trying to find product-market fit, you probably have bigger fish to fry.

The biggest tech debt I've struggled with always comes from the same patterned source. You think a system is intended to have a certain feature set for a certain user base, and so you make a set of architectural decisions to match. The world moves on quickly, you adapt, pivot, whatever you want to call it, and suddenly your carefully made architectural decisions are no longer correct or wise for your new scenario, but there is no time to go back and have a "do-over".

So in essence, failure to be able to tell the future, paired with a constant need to move forward, is the source of the problem. Both of those things are going to keep happening to startups forever.

3 comments

In other words, it’s not a problem to be avoided, but rather a problem to be embraced.

If this is indeed true, how does one fully embrace it? Optimize for easy refactoring? What does this look like in practice?

No, I wouldn't put it that way. It's not a problem to be embraced, it's just another variable to manage.

Just as there is the trilemma of "better, faster, cheaper: pick 2" there are others as well. Speed and tech debt are balanced against one another. It's not that I think we should embrace tech debt. I actually rather hate it. But sometimes I'm willing to trade to get to a business objective that is not related to technical elegance.

In terms of how to pay it off, it may sound like a cop-out but it just depends. The core problem is that things are always changing and we can't predict the future. So staying supple and flexible I think is the way to go. Pay off as much of it as you can when you can, but I think the main point for me is to keep the eyes on some business objective (where tech debt appears as only one variable in the equation) -- the main point is not to keep your eyes on the tech debt all the time.

I’ll give my perspective (CTO at a small start-up), and I’ll start by saying that I fully agree with the GP.

Technical debt for us manifests as new cards in the backlog (and/or TODO comments in the code), because when we cut corners we usually have the presence of mind to flag it. I find that as long as these cards don’t rise to the top of the backlog, then it’s safe to ignore them. It means that you can afford to pay the interest of that technical debt.

If however you find yourself repeatedly thinking “To do this feature I really need to fix this issue first”, well then it might be time to repay some of the principal, so to speak.

In practice, the first time we build a new type of feature, we’ll do it quickly and dirtily and accept that it may be throwaway code. If we end up doing the same thing a second or third time, then we’ll refactor. But then we’ll know there’s a need to do it cleanly, and we’ll have two or three examples from which to try to generalize what the architecture should be. On the other hand, we really try to avoid over-engineering and premature optimization. If it’s good enough for Donald Knuth...

It is a tool to leverage when appropriate.

You can write tests when prototyping or you can borrow a little tech debt and skip the tests during the prototyping phase. At this stage it is like a line of credit.

Then, as the feature/product starts to take shape, you can pay down the debt by writing the tests and refactoring the code to be simpler. This would paying off the line of credit.

Or, you can roll the line of credit into a term loan and move on. This will increase your long term debt.

I'm not in a startup, but I always try to budget ($/time) for 20% overhead to undo stuff technical debt. When/where that isn't possible, it just gets harder and harder to fix.

These issues are "perfect is the enemy of the good" scenarios. If you don't have "technical debt" or something you don't like or got 80% right, that is a far bigger problem. How to manage really depends on your scenario.

This is where most of what 'debt' i manage comes from. We have a very large ecosystem of interconnected systems many of which have roots 15 years in the making. While the design decisions made back then were valid the expectations of our clients and industry have of course evolved. We're a small team and it's impossible to stop and rebuild the system so I try to manage it now by building services we can plug into the various systems. It helps reduce the scope of future debt and often allows updates without the stop and rewrite nightmare. Doesn't work for every case but it has helped and hopefully whomever I pass this on to will find it easier to manage.
This sound very familiar, the software development company I work for uses at least 10 different platforms internally across Linux, Windows & Unix...SQL server, mysql & oracle dbs. 10yr old PHP customer support systems, 8 yr old jboss & tomcat servers with deserialization vulnerabilities at every turn, .NET V3 testing tools, mixed up with a sprinkling of modern platforms all talking to each other through rest, direct DB connections, microservices & a hearty dose of black magic. Management ignores vulnerability reports due to mostly non-technical backgrounds. Most documentation is at least 3yrs old, when 80% of dev team sacked & outsourced to India. Mostly keeping the lights on now during an acquisition...can't wait to debrief the new owners. Have acquired some great experience in investigating technical debt in the process which will hopefully be useful in the future.
Technical debt is less analogous to financial debt, and more like an anchor. You want to move fast to keep up with the market, but your debt slows you down.
That sounds exactly like financial debt to me. Too much (financial or technical debt) is an anchor, healthy amounts of debt result in not being able to do everything / having some limitations but overall you're in a good place, and no debt probably means it's time to actively pursue opportunities. No?
With technical debt, each new feature now costs $nx where n is the multiplier for the accrued technical debt. A feature that you need in 2 months to support a market shift now needs 4 months to complete due to technical debt.

The issue with this oversimplified formula is that you cant accurately determine which debt affects which features. For some features it could be zero, and others it could be 100.

However, I do agree that all teams should be carrying an amount of technical debt to be healthy. It shows a certain quality of decision making to balance it well.

All too often though it becomes an excuse to procrastinate and that might as well be gambling.

I see this as interest on debt. When you have $1K of credit card debt, a $500 payment goes a long way. When you have $20,000, that same $500 is mostly covering interest, so you pay down the principal slower.

The additional complexity and work that comes with tech debt is the interest you pay on it.

Not really, I have seen whole features dropped. Like payment processing, business had an idea to use stripe, 6 month later all code deleted because our business model was not for people who would pay with stripe.

We also pivoted other application where only database stayed roughly the same. Loads of legacy stuff was still there but it is going to be phased out soon.

So I have seen situations where tech debt was never paid back. I have also seen one guy that is not paying me back my money, but I still remember he owes me. Some tech debt will go into oblivion in next year or two...

That anchor effect of being slow is what I meant by "installment payments". Financially, if you have enough debt you can make $300k a year and still live in a crappy apartment, because all of your income (in this weird metaphor, that's your available dev hours) are being spent on debt (fixing things that wouldn't have occurred had you not had the debt)
I will concede some points in your favor for that. I will counter with the fact that almost no CTO will change there deadlines accordingly, which leads to team burnout/turnover.