[Zababa]

> and tests should explain why it's doing it

I was going to say "So for performance "hacks" there should be a clean implementation that's benchmark against the current implementation for example?" as a way to disprove what you said, but while writing it I realized that it may actually be a pretty good idea.

[dmurray]

Yes, there should pretty much always be this.

You've just written some messy unclear code for performance reasons, so it deserves more thorough tests than average.

And you've already written the clean version anyway, since you hopefully didn't write the optimized version until after you profiled the slow one. So it's not even significant extra work to turn that into your test suite.

[Zababa]

This could also be a great way to check if your optimized version is still necessary with new versions of your language implementation. The more I think about it, the more it seems like a great idea.

[dmurray]

I wouldn't generally throw benchmarking into the unit test process, because it can be brittle. What if something else clobbers the CPU on your test machine at just the wrong time. What you really want from your unit tests is for them to run quickly, so you run them as often as possible, and reliably, so any failure means something is wrong. I'd use the slow implementation just for checking correctness.

But separately, if you want to add a performance benchmarking process (great!), you have a reference baseline implementation for free.

[Zababa]

Maybe not directly in the unit tests that you always run on your machine, but having them around could be a good idea.