Hacker News new | ask | show | jobs
by geraltofrivia 22 days ago
> You'd essentially need to write a single test suite that works for both projects in each pair to compare fairly.

That's pretty much what we did. We start with a repository, and

- (vibeclean pipeline) make changes to clean analyzer issues (some of them involve moving big code chunks around to reduce cognitive complexity), and tweak the existing test sets when needed.

- (slopify pipeline) inverse, add analyzer issues and increase complexity, add dead code etc.

In both cases, we ensure that the test coverage remains the same, and the tests are passing, before we start our experiments.