[Wronnay]

I can't really agree with the testing part. I worked with many projects where I came across Issues which definitely could have been prevented by more automatic unit / E2E tests.

I imagine the engineers who wrote the code for the Boeing 737 MAX MCAS or the Tesla engineers who accidentally wrote log files to the internal flash thought the same. Maybe another team member could show the author why these tests are important...

[rob74]

TBF, he does write "I do see the importance in many of the things I just listed [one of them being testing], but they sure do take the fun out of the act". And I, for one, agree with that. Testing is important, but in practice it's often just another dumb metric (e.g. achieving x % code coverage), which however doesn't say anything about whether the tests are actually helpful or just call the code so it's "covered"...

[resonious]

The problem here (in my mind) is that hindsight is 20/20, so you can always look at a bug and say that a test could've prevented it. But in the author's case, it sounds like they still have bugs despite all the tests, so I guess the people writing the tests just aren't very skilled? And if that's the case, couldn't a more skilled engineer just write better code to begin with?

I totally get tests as a way to prevent known incidents from happening again. But tests as a way of preventing unknown issues seems dubious to me.

[spc476]

> But tests as a way of preventing unknown issues seems dubious to me.

True. At my previous job, I used a new C compiler [1] that revealed a bug due to undefined behavior [2]. It was a two-line fix, but my manager at the time refused to accept the fix unless I 1) made a test that showed the error, and 2) write tests to show other instances of this type of bug. The test would have been useless since I used an unsanctioned C compiler, and (due to the nature of the bug) I'm not sure if static analysis would have found the bug anyway.

[1] Test driving the latest Ubuntu Linux installation for development.

[2] The code worked as expected on the business-supported C compilers (it's undefined behavior, anything can happen). Even valgrind didn't catch the error (we were using that at the time) because of the way the code was generated.

[potatopatch]

The frog on a dissection table approach to software development seems like it is probably a primary cause of incidents involving lost surgical tools in production.

[Kinrany]

Say more?

[potatopatch]

In traditional development you didn't have one organ in a harness, then 3 organs in a harness, etc, having modified them to have no thickness that could hide something from a unit test. That's as different from the functioning frog as the dissected one.

There are endless permutations that are possible in this modern ideal that are all correct for some or all testing purposes, during all test runs, and in every system the average project participant will ever see up close.

They are wrong in production, vastly outnumber correct production configurations, and confuse any discussions about the attempt to port things to production as they are more canonical to the group than production itself.

That's not to say there aren't ways to do this style of multiple test types correctly, only that I think it was done with more care at each level instead of less as layered redundancy needs particular properties or you get holes that all line up, configuration for tests that make it to production, etc.