[kelnos]

Sure, but behaviors that never have a bug or regression don't get a test. Software of this kind of complexity has all kinds of behavior that has never been broken, and doesn't have a specific test written for it.

Getting an extensive test suite passing is certainly orders of magnitude better than having no test suite at all, but it still doesn't tell you as much as you need to know. I would absolutely never trust an LLM Postgres rewrite (in any language) in production based on "only" Postgres's test suite passing.

[bob1029]

> Software of this kind of complexity has all kinds of behavior that has never been broken

This space of things is astronomically larger than the space of things expressly covered by any test suite.

"Program testing can be used to show the presence of bugs, but never to show their absence." -Edsger W. Dijkstra

[w4der]

I've also seen situations where a customer reports a bug, the fix breaks some regression, and the updated behavior to work around the fix breaking the regressions turns into an undocumented feature.

[derdi]

How do you break a regression? A regression is breakage. Are you one of the people who use "regression" to mean "regression test"? Did Codex learn this from you? I hate it.

[w4der]

Yeah, sorry, it was a codebase with _only_ regression tests dating to the 80s, so we called everything a "regression".

[mbrock]

The same basically holds for proofs in the absence of coherent global correctness criteria like, say, confluence and normalization for a lambda calculus, or soundness and completeness for a logic.

Fable's napkin estimate of the effort required to produce a passable reference semantics for Postgres, which would involve novel discoveries in denotational semantics of concurrent transactions and so on, might be in the ballpark of 30–60 years of PhD level work.

So realistically I think the only way to validate a Postgres implementation involves differential testing, fuzzing, acceptance test suites, etc. And still you'll have bugs that need to be hammered out the good old fashioned way.

[gblargg]

Or even a human rewrite merely because some language is the current fad. A rewrite in a different language should be done for very good reasons, to solve problems that are bigger than the costs of all the bugs that will be introduced.

[jghn]

Perhaps before embarking on one of these rewrites the first step should be a heavy round of mutation testing and property based testing. Contribute any new testing code from this back to the original project. And *then* embark on the rewrite.

[vitally3643]

If that's your concern, then your argument becomes "software should never change". Why dare patch any bug ever? It might be load-bearing in some unknown, undocumented, unsupported workflow somewhere in the world. No test imaginable can catch that apart from the scream test.

There are reasonable arguments against language ports, but this is not one. You're making an argument against code changing at all ever.

[gb2d_hn]

Agreed.And a rewrite in another language creates a high probability of a change in behaviour