[grog454]

> Are there any other products with such a small team and a huge userbase?

My game Nebulous was 1.5 devs (one full time one part time) and multiple millions of MAU. 9.5 years later it's still going well.

> when the business requires more coding and technical debt comes how do they manage it

Delete bad code. Replace with good code. Sounds simple enough but in my experience at mega and mid corps, step 1 is almost never done. Whether that's because of ego or chasing local optima I'm not sure - probably a mix of both.

[cbm-vic-20]

It's due to fear. Fear of breaking something that may depend on that bad code. Test automation rarely covers every possible case, and nobody wants to be on the hook when some code changes cause other stuff to break.

[klabb3]

Can confirm. One of my proudest moments was deleting thousands of LOC of copy-paste-modify garbage. However I introduced one bug that broke another team which used an undocumented feature. It was fixed soon, but yeah, still not great. And very few people would have taken that on, I was not a career chaser.

Some would say it’s the other teams fault for not adding a cross-test against my teams code. And while that would have solved it, some things are hard to test. Even in companies who have good testing standards some things are still hard-to-impossible to test. In my humble opinion tests are great if and only if they are hermetic and fast. Unfortunately, the important things that can go wrong are usually the least testable.

In either case, in a non-perfect world (ie ~all large companies and most small ones) people optimize for not breaking things, and there’s a solid argument for that being a local optima, both for short term stability and career wise.

[giantrobot]

> In my humble opinion tests are great if and only if they are hermetic and fast. Unfortunately, the important things that can go wrong are usually the least testable.

Integration tests are hard. A lot of time it's because deployment is very seat of the pants. Even with tightly managed deployment the test environment needs to be representative of the production environment. Just setting that up is time consuming and expensive. Then actually doing tests where the test environment has useful amounts of instrumentation without major performance or behavioral penalties.

[shadowgovt]

And for all of the benefits of process, I have never met a level of documentation, verification, or testing that matches the advantages of having the entire code base originating from one mind.

Generally this is not tractable because it cannot scale. But there are certain applications where it scales fine.

[mattgreenrocks]

I don't think we need nearly as many devs as we think. We do need someone who is taking a hardline approach on limiting the amount of scope tackled at once, and then fewer devs that are downstream of that.

It is hubris to think that every problem admits the same solution, namely, throw as many devs as we can at it and hope for the best. But business isn't really known for being reflective.

[swiftcoder]

> Generally this is not tractable because it cannot scale

The real question here is scale in terms of what? Because a lot of folks are out here trying to scale people/careers, not software.

It's extremely noticeable at BigCorps. Why do we need to scale this project from 3 -> 30 -> 300 developers? Because that's the number of reports to promo from Manager I -> Manager II -> Director

[atomicnumber3]

That, and most orgs simply do not reward or even pretend to care about these kinds of improvements. If you delete bad (but working) code, and replace it with good (and, let's assume best-case scenario - also working) code, what has actually changed for the business?

Nothing. Except that in 3 years the junior dev that gets a ticket about doing something in this area will come in and not notice the code isn't a dumpster fire. Or, in 3 years, you won't notice that you didn't have to optimize this code a year ago.

What they do notice is that you were insisting on working on some mumbo jumbo and ok good they're done now they can actually work on something useful. Haha aren't these devs quirky? Sometimes they take a few days and work on something weird, and all the other senior devs nod and salute solemnly and I'm too scared to ask for more details, but they don't usually take too long so let's just indulge them for a few days to keep them happy so they don't leave too.

It takes a very, very deeply engineering-first org to really cultivate this intentionally. And similarly it seems like succeeding as a startup requires at least a decent amount of shipping some shit code fast so you get a series B, so usually you don't start in this posture and never shift into it before it's far too late.

And also unfortunately, devs often _do_ spend time optimizing/refactoring personal pet peeves as opposed to things that might have a good chance of mattering. I once saw another senior dev spend a week optimizing string allocations on our hot path. Our owner loves people who can do this kind of stuff, so it got a lot of praise. The microbenchmarks looked great, pretty graphs. Users noticed nothing, the actual metrics we track literally did not change, and now the already-complicated hot path is decorated with some contorted string-allocation-avoiding warts here and there and the next person to go in and change the code is _definitely_ going to keep doing that pattern, for sure. Meanwhile our oauth flow is still a tortured, unloved, twisted writhing mass of pain and suffering that prints bug reports like CVS receipts.

So... extraordinarily difficult to intentionally cultivate a culture that does this judiciously.

[block_dagger]

One of the most rewarding aspects of my previous career at a company spanning from a startup to an IPO and beyond was deleting bad code and replacing it in a massive Rails app that was touched by hundreds of devs in a high churn environment. I also took on fixing massive schema inefficiencies that had a lot of risk of breaking nearly every other team's flow. It took a lot of careful work and communication across multi-year goals that I managed, mostly alone. I was allowed to do this by a few early folks who believed I was doing a good service for the company in the long run but kept hinting it was a bad career choice for me personally. I believe I was eventually let go for making these massive improvements instead of adding that green button that the new Product guy wanted. No regrets.

[BubbleRings]

I think I’ve taken some considerable career hits for that kind of attitude, but mostly no regrets here either. But I think I was affected by how, once you leave that company, your contribution to the effort can seem kind of gone, gone gone. That’s part of why I came back to trying to create a physical invention that someone might care about. Something for the grandkid to put on his mantlepiece and say “my granddaddy made this and patented it and (hopefully) it was the start of his big company.”

[gerad]

> Meanwhile our oauth flow is still a tortured, unloved, twisted writhing mass of pain and suffering that prints bug reports like CVS receipts.

Wow, this line is a keeper. This whole comment is so insightful. Reminds me of how awesome HN can be sometimes.

[fmbb]

Good code and bad code are not objective values.

I have worked with many people that spend days replacing good code with bad code because they are ”paying down technical debt”.

[atomicnumber3]

This is a good point, but I think it's mostly a precondition to having the luxury of the problems I was describing. If you can't even broadly agree on what is good vs bad code, your engineering org has deeper problems. You don't even need substantial agreement, just enough to identify what the genuine problem areas are, vs what's just not how someone would've written it themselves.

[fmbb]

You need to agree on the problem, not the problem areas.

Then someone proposes a solution for the problem, and you review the code to see that it solves the problem. If the solution is not bad and the problem is big, you can let them merge it.

[nasmorn]

I once worked with a guy that thought all working code was good code. I literally showed him a massively improved refactored version and he basically just shrugged. Once he left our velocity went way up

[BubbleRings]

You should write a book on this stuff.

[mritchie712]

it's always fear for me. I'd rather leave it commented out for a decade or so to be safe.

[Feathercrown]

I'm a monster, I see code that's been commented out for more than 2 months or so and nuke it unless I know it's needed. We have Git, it'll be fiiine

[astrospective]

I bring this up in PRs, as I know with my own experience I don’t always mean to commit it commented out, sometimes it was just for testing, or it should have been deleted after I was done refactoring.

[radlad]

Right. The only time I sometimes leave a commented out line of code is if it's temporarily broken and will be uncommented soon; or if it's by-far the most obvious way to do something, but does not work for some reason, and then there's a comment above about why not above.

[devjab]

I never understood commenting out code when you have version control. I get why people do it, I’ve done it myself and then two days later been confused which of the 3 commented out function was actually the most recent. It’s infinitely more clear from the version control since the history is there for you to zoom through.

[wink]

You would find it if you knew it was there.

I only leave it commented out because it has a reason (they all say that, right?).

I suppose the best way would be to provide a meaningful comment "This is the place where 15 lines of coded finally found their resting place, deleted after the bug they solved was eliminated elsewhere".

But in reality, I've never seen a single of the "we could find it it in git" ever actually find it in git.

[devjab]

Well yes, but you’re not leaving commented out code for anyone other than you. A “sane” git structure will automatically decline your pull request if it contains commented out code.

I say “sane” because I know a lot of places probably allow you to do it. You really don’t want to pollute a code base like that though.

[swiftcoder]

> It’s infinitely more clear from the version control since the history is there for you to zoom through

Only if you already know it is there. There is like... zero history discoverability built in the git. and git's historical search story is pretty bad too.

[devjab]

Well, who other than you would need to know about your commented out code? I’m not suggesting you keep the commented out code as part of your git history, that would never be allowed through a pull request. The changes will be there in the history though, I doubt you’ll need to go back for them, but you could.

[bryanrasmussen]

>I never understood commenting out code when you have version control. I get why people do it,

I'm not suggesting that one should comment out code but I never understood why if they have version control and there is a new bug a significant amount of devs do not actually go through the history to find the working version.

Now - after writing the above and thinking while writing, I am going to suggest scenarios when you want to comment out code.

You write a great bit of code that does not work, you find out it is because API X is not correctly implemented, therefore you have to do a workaround kludge until API X is correctly implemented.

What do you do? I think the optimal solution there would be to comment out the great bit of code above the workaround kludge with a comment - API X has a bug, I have made a bug report out on the API X working group. I suggest checking periodically and then getting rid of the kludge and moving this code in. Dated, so that people can see when they should check.

It's true I have seldom had the pleasure of working with people where this optimal way of doing it would have been at all useful, but it has happened maybe twice over many years. Probably it will only help you out when you see the comment over the coming months, or when you get an email that your bug has been fixed.

I'm sure you can imagine other similar scenarios in which commenting out the code makes more sense than using version control.

[block_dagger]

I think a lot of devs prefer the hover-to-blame feature in their IDE vs searching through history on GitHub.

[noprocrasted]

> Delete bad code. Replace with good code

Your points are valid but there's also the issue that the more developers you have the more communication overhead there is, which makes large changes to the codebase hard/impossible.

With a handful of devs you can jump on a call, brainstorm for an hour or two and come to a mutual agreement, then one can submit a several-thousand-line PR refactoring the whole thing and nobody would bat an eye.

This kind of coordination is impossible in larger teams, if anything just because everyone is busy and can't afford to spend a couple hours brainstorming + subsequently get acquainted with the new code, but also because the more people the more opinions and mismatched incentives (bad or overly complex code might imply busywork which some people thrive on, so refactoring it to no longer require said busywork is a downside in their eyes).

[gregors]

>>>> This kind of coordination is impossible in larger teams,

And that's the real reason service architecture took off

[dowager_dan99]

I go further and simplify: "delete code" whenever and where possible. The term "Tech Debt" is really overloaded; I think the idea that "all code is liability" is better for framing the issue and strategies.

[BubbleRings]

It is so good to see someone say that. I don’t code anymore, but as a systems engineer on different (often troubled) projects, I started developing a bit of a specialty in deleting crufty old collections of files. Sometimes multiple terabytes in a day, directories sitting around looking like they might be important, in some random corner of storage.

You have to be good at your job, good at the specialty, and more interested in doing the right thing for the company (and more irritated at the stupidity of the files being there 10 years after they were needed) than you are at looking productive to management. Management does not want to hear “well there was a directory structure of two million files that was a backup of a Linux machine from 8 years ago, I spent two days extracting the dozen files that we might need some day, getting the okay to proceed, and deleting the files.”

[Consultant32452]

I have no incentive to delete bad code and replace it with good code when doing megacorp work. For things I own, it's situational.

[moritonal]

Fleet commander or I'm guessing .IO?

[grog454]

.io. There have been more Nebulous's since I last checked :)