[Adutude]

So this appears to be, the very old, software that was pushed to production before it was ready story. I have to wonder if there wasn't some Marketing/PR/Mgmt person(s) that pushed the engineers to release the software before it was ready or was it the result of lack of due diligence on the part of the engineers, who couldn't wait to get their code running in the real world.

>We did implement checks for what seemed to us as more common failure scenarios, but the devil here was that this one first appeared during the run and we did not cover it at the analytical analysis stage.

I little more time in testing for the edge cases could have helped avoid this expensive fail.

You have to test for much more than just the "common failure scenarios", when you engineer solutions where failure could be very expensive in terms of money and lives.