Hacker News new | ask | show | jobs
by masklinn 3790 days ago
> You can only test hypotheses of the form "A is exactly like B" - no bug fixes are allowed, because they will show up as differences.

Of course they will. That's a feature. The system doesn't intrinsically know whether a change in behaviour is a bug added or removed, it can only report that there's a difference in behaviour, it's your job to investigate whether the control or the experiment is correct.