[floxy]

>the time to evaluate whether a test is fair or not is generally not when you're failing the test.

...well, this is a technical forum, and mechanical engineers get these types of questions all of the time. "Hey, the machinist accidentally machined this wall 0.010" too thin. Is it OK to use?" Then you run an analysis, and report something like, no, that's too thin, scrap it. Or, yes, because of X, Y, and Z, this one is acceptable. And maybe this means that you can update the drawing to use a looser tolerance. Or maybe it just means that this one time it is OK, because there is another expensive process that you can do to the part to salvage it. Then there is a procedure to track this non-conformance, until it becomes conformant again. And it becomes part of the permanent record. The missing piece of the article's puzzle is whether this was a critical dimension that should have passed some sort of inspection process. Maybe it should have been a critical (inspected) dimension, but wasn't called out as such on the drawing (a documentation error). Or maybe the inspection was called out, but wasn't done. Or the inspection wasn't done correctly (i.e. inspector reported that it met the tolerance). Or the failed inspection reports were ignored. Or the non-conformance was reported, and an analysis was done and was shown to be fine, but that paper-trail has disappeared. Or in fact all the paper-work is in order, but for some reason wasn't available to the person informing the author of the article. Lots of different rabbit holes to go down here, but we don't have much to go off of.