Hacker News new | ask | show | jobs
by rob74 235 days ago
If I understood correctly, the test suite they are using are the tests used for the original coreutils. Apparently, the behavior that led to this bug wasn't covered by the original test suite (and it looks like the original coreutils just got the behavior "right" without it being tested), so the uutils guys added new tests to cover this after the bug was found.
1 comments
pclmulqdq 235 days ago
That makes sense. However, I am generally biased against making significant changes in software (especially rewrites) without also beefing up the test suite, though.
link
junon 234 days ago
How could they beef it up if they don't know the problem is there?

You can write frivolous tests all you want; bugs are a part of life. They'll occur even if you test thoroughly, especially in new software. It's the response that matters.

link
pclmulqdq 234 days ago
They could use the description of what the software is supposed to do and an understanding of the current code to figure out how it should work and what edge cases need to be tested. They can also test random inputs and so on. When you write new software, you do this. When you add features, you do this. When you do a rewrite or refactor, you should also do this.
link
junon 234 days ago
That's so, so much easier said than done.
link
soleveloper 234 days ago
Is it? I hope I won't step on somebody's else toes: GenAI would greatly help cover existing functionality and prevent regressions in new implementation. For each tool, generate multiple cases, some based on documentation and some from the LLM understanding of the util. Classic input + expected pairs. Run with both GNU old impl and the new Rust impl.

First - cases where expected+old+new are identical, should go to regression suite. Now a HUMAN should take a look in this order: 1. Cases where expected+old are identical, but rust is different. 2. If time allows - Cases where expected+rust are identical , but old is different.

TBH, after #1 (expected+old, vs. rust) I'd be asking the GenAI to generate more test cases in these faulty areas.

link
pclmulqdq 234 days ago
"You have to catch everything" is much easier said than done, but "add at least one new test"? Nominally the people doing the rewrite should understand what they are rewriting.

Usually, the standard a rewrite is held to is "no worse than the original," which is a very high bar.

link