[tonyarkles]

Last week I stumbled onto a bug in a sensing system that has lived in our codebase for at least 16 months and wasn’t ever triggered. In being vague but the system has been used pretty heavily in a whole bunch of different environments attached to several different host (mechanical) machines. What tickled it? A counter wraparound in 3rd-party FPGA logic at exactly the wrong moment.

And per Murphy’s law, it happened for the first observed time in a relatively high-stakes situation while there were a lot of eyes on it. Naturally.

[DoctorDabadedoo]

I feel you. This week we found a race condition induced segmentation fault in our lidar drivers (by Sick) that have always been there and the only reason we came across it was because we had to customize them and increase the poll frequency for a specific use case. Like you, high visibility and during a crunch.

[rcxdude]

AKA the demo effect: if you want to show something working (like to a customer, investor, boss), it won't. If you want to show something not working (like to a supplier or co-worker for refund/debugging), it will.