[layer8]

The problem with complex legacy codebases is that you don’t know about the myriads of edge cases the existing code is covering, and that will only be discovered in production on customer premises wreaking havoc two months after you shipped the seemingly regression-free refactor.

[ljm]

It helps if tests are well written such that they help you with refactoring, rather than just being the implementation (or a tightly coupled equivalent) but with assertions in it.

Rare to see though. I don't think being able to write code automatically means you can write decent tests. Skill needs to be developed.

[danparsonson]

And the challenge with writing tests well is that it might be difficult or impossible without the refactor...

[ljm]

I disagree. There is no code that can't be tested. There is certainly no code that has to be changed so it can be tested.

The only reason you would decide code is untestable without touching it is because it doesn't seem worth the effort, so you test the thing that calls that code instead and mock it out.

At the highest level it is all inputs and outputs after all. Your spaghetti app can still be tested end-to-end, which will cost you the most but will work.

We're talking about SaaS and web apps and shit here of course, because we're on HN where downtime on Cloudflare equals a day off work. In other industries you'd be running full blown simulations and proofs and maybe even inventing new mechanisms of testing so you can get a hard project off the ground. Maybe even fabricating materials.

[danparsonson]

> ...the challenge with writing tests well...

Sure, you can test more or less anything given sufficient time and force, but a massive fragile harness wrapped around the system under test is not what I mean by 'testing well'.

Also there are other things in software besides web; unit testing code that is entangled with desktop UI can be difficult or impossible without changing the original code. Try testing a WPF app that talks directly to the Dispatcher when there is no UI thread, for example.

[mehagar]

I agree. This is one area I'm hoping that AI tools can help with. Given a complex codebase that no one understands, the ability to have an agent review the code change is at least better than nothing at all.

[UltraSane]

If you save a log of input on the production system you can feed it to old and new versions to find any changed in behavior.

[nijave]

You can infer based on code coverage. If coverage is low, tests are likely insufficient and change is risky