Hacker News new | ask | show | jobs
by charles_f 110 days ago
Write tests. Most likely those 300k lines of code contain a TESST folder with 4 unit tests written by an intern who retired to become a bonsai farmer in the 1990s, and none of them pass anymore. Things become much less stressful if you have something basic telling you you're still good.
4 comments

The problem with complex legacy codebases is that you don’t know about the myriads of edge cases the existing code is covering, and that will only be discovered in production on customer premises wreaking havoc two months after you shipped the seemingly regression-free refactor.
It helps if tests are well written such that they help you with refactoring, rather than just being the implementation (or a tightly coupled equivalent) but with assertions in it.

Rare to see though. I don't think being able to write code automatically means you can write decent tests. Skill needs to be developed.

And the challenge with writing tests well is that it might be difficult or impossible without the refactor...
I disagree. There is no code that can't be tested. There is certainly no code that has to be changed so it can be tested.

The only reason you would decide code is untestable without touching it is because it doesn't seem worth the effort, so you test the thing that calls that code instead and mock it out.

At the highest level it is all inputs and outputs after all. Your spaghetti app can still be tested end-to-end, which will cost you the most but will work.

We're talking about SaaS and web apps and shit here of course, because we're on HN where downtime on Cloudflare equals a day off work. In other industries you'd be running full blown simulations and proofs and maybe even inventing new mechanisms of testing so you can get a hard project off the ground. Maybe even fabricating materials.

> ...the challenge with writing tests well...

Sure, you can test more or less anything given sufficient time and force, but a massive fragile harness wrapped around the system under test is not what I mean by 'testing well'.

Also there are other things in software besides web; unit testing code that is entangled with desktop UI can be difficult or impossible without changing the original code. Try testing a WPF app that talks directly to the Dispatcher when there is no UI thread, for example.

I agree. This is one area I'm hoping that AI tools can help with. Given a complex codebase that no one understands, the ability to have an agent review the code change is at least better than nothing at all.
If you save a log of input on the production system you can feed it to old and new versions to find any changed in behavior.
You can infer based on code coverage. If coverage is low, tests are likely insufficient and change is risky
The best time to write tests was 20 years ago. The second best is now, provided you've applied to all the companies with better culture.
I've been working on react and react native applications professionally for over ten years, and I have never worked on a project with any kind of meaningful test coverage
I have not seen tests in any code base I worked on in the past 20 years. I have noticed that there is some kind of sanctimonious demeanor to quite a few people that advocate for tests (on comment boards). I find the reactions to discussions on tests fascinating because it seems to elicit very strong opinions, sort of a "do you put your shopping cart back" kind of topic, but for programmers.
I find that fascinating, because interacting with the tests in our codebase (both Python and JS) answers a _lot_ about "how is this meant to work", or "why do we have this". I won't say I do test-driven development, at least not very rigorously, but any time I am trying to make a small change in a thing I'm not 100% familiar with, it's been helpful to have tests that cover those edge cases. :)
People who advocate “writing tests” never admit any of the costs and difficulties of automated output checking.

As you change your codebase you will experience lots of “failures” that are not failures. You still have to burn your time investigating them.

Many checks will require elaborate mocking or other kinds of setup, that give lie to the claim that designing them is simple and straightforward.

I've checked the stats, the previous app I've worked on has 31% reported coverage and I think the actual value is higher, with coverage of most of the critical paths. But it's been a lot of work and the engineering hierarchy is supportive in adding time to manage the existing tests and test the new features.
over 20 years, many stacks, and same
> I have never worked on a project with any kind of meaningful test coverage

That says more about you and the care you put into quality assurance than anything else, really.

Have you ever worked at a place where you were put on an existing codebase, and that code has no tests? Have you ever worked at a place where, when you try to fix that, management tells you that they don't have the time to do so, they have to crank out new features?

Is ipsento606 working at such a place? I don't know, and neither do you. Why do you jump to the conclusion that it's their personal failing?

> Have you ever worked at a place where you were put on an existing codebase, and that code has no tests?

Yes.

Then I added tests. Now the codebase has tests.

Funny how that works.

> Have you ever worked at a place where, when you try to fix that, management tells you that they don't have the time to do so, they have to crank out new features?

Yes.

I then added tests that covered my features. Now the project has tests.

Funny how that works.

Presumably you mean me, and every current and former team-member I've ever had? If so, you're talking about hundreds of engineers.
> Presumably you mean me, and every current and former team-member I've ever had?

Yes, I mean you.

You don't need hundreds of engineers to have a test suite with meaningful coverage. You need yourself and willingness to do the job.

Just adding happy path tests can get you significant coverage. Why did you failed to do this?

Blaming others for the problems you created and help create is not helpful. You can post PRs, can't you?

A difficult prerequisite for that might be untangling a very unatomic codebase into testable chunks. And to determine a feasible "level of abstraction" to write tests for. Testing a full pipeline of a numerical library might be as impractical as testing super tiny functions, because both won't allow you to really work on the codebase.