[zephen]

There has always been a laissez-faire subset of programmers who thrive on living in the debugger, getting occasional dopamine hits every time they remove any footgun they previously placed.

I cannot count the times that I've had essentially this conversation:

"If x happens, then y, and z, it will crash here."

"What are the odds of that happening?"

"If you can even ask that question, the probability that it will occur at a customer site somewhere sometime approaches one."

It's completely crazy. I've had variants on the conversation from hardware designers, too. One time, I was asked to torture a UART, since we had shipped a broken one. (I normally build stuff, but I am your go-to whitebox tester, because I hone in on things that look suspicious rather than shying away from them.) When I was asked the inevitable "Could that really happen in a customer system?" after creating a synthetic scenario where the UART and DMA together failed, my response was:

"I don't know. You have two choices. Either fix it where the test passes, or prove that no customer could ever inadvertently recreate the test conditions."

He fixed it, but not without a lot of grumbling.

[Verdex]

My dad worked in the auto industry and they came across a defect in an engine control computer where they were able to give it something like 10 million to one odds of triggering.

They then turned the thing on, it ran for several seconds, encountered the error, and crashed.

Oh, that's right, the CPU can do millions of things a second.

Something I keep in the back of my mind when thinking about the odds in programming. You need to do extra leg work to make sure that you're measuring things in a way that's practical.

[crystal_revenge]

I've recently had a lot of fun teaching junior devs the basics of defensive programming.

The phrasing that usually make it click for them is: "Yes, this is an unlikely bug, but if this bug where to happen how long would it take you to figure out this is the problem and fix it?"

In most cases these are extremely subtle issues that the juniors immediately realize would be nightmares to debug and could easily eat up days of hair-pulling work while someone non-technical above them waiting for the solution is rapidly losing their patience.

The best senior devs I've worked with over my career all have shared an uncanny knack for seeing a problem months before it impacts production. While they are frequently ignored, in those cases more often then not they get an apology a few months down the line when exactly what they predict would happen, happens.

[zephen]

> While they are frequently ignored

And this is the reason I spent most of the latter part of my career in chip companies.

Because tapeouts are _expensive_, both in dollar cost, and in lost opportunity cost if the chip comes back broken.

So any successful chip company knows to pay attention to potential problems. And the messenger never gets shot.