[mannykannot]

Optimizing for the test, in a way that does not necessarily have any relevance for real-world performance, is much more of a possibility for software than it is for physical things. This is because of the fractal/chaotic nature of software: small differences can result in wildly different outcomes.

NHTSA testing has been developed over a long time, and in conjunction with studying both real-world and laboratory crashes, so I imagine they are quite realistic and representative. The one way that they equivocate the tests that I am aware of is to divide vehicles into weight classes, but this would be difficult to take advantage of.

[danielbigham]

A bit off topic, but I've never seen "fractal" and "chaotic" conjoined before. If you feel like expanding on that I'd be all ears.

[mannykannot]

On reflection, the 'fractal' bit isn't relevant here.

The fractal nature of software is the way that the abstract specification - concrete implementation dialog appears at all levels of detail.