[coldtea]

>Which run, I must add, on effectively infallible hardware.

Keyword: effective. Hardware is also built on top of components with error tolerances. What do you think ECC memory is for? Or why chips have "yields" and parts that were "turned off" while shipping? Or how thermal throttling happens? Or CPU clocks, which have jitter to be corrected, and tons of other examples, all the way to transistors and capacitors.

And let's not even get into HDs and SSDs.

[Joker_vD]

My point is, when you go from hardware to software, there is a noticeable change in attitude and corresponding drop in reliability. Inside hardware, there is a lot of effort to detect and correct glitches and provide reliably correct behaviour; in software, the most common assumption is that the hardware will just work as intended. How many filesystems store checksums? Even back in the days when HDDs were way less reliable? And ECC exists in no small part because writing software that's reliable in the face of memory-corruption (at the very least can detect such corruption) while is possible, it's hard, costly, and produces less performant software (obviously).

Anyway, a larger point of my comment is that when we go from traditional software to AI, there a) seems to be an even bigger drop in reliability, b) the attitudes are even more cavalier, c) the software people largely lack most of discipline/knowledge required to build more reliable system from less reliable ones — because they spent most of their careers doing exactly the opposite: building software that they know will work incorrectly even on perfect hardware, and being fine with it. "You just tend to accept the risk", my ass. Yes, when it comes to software, people do this, but when it comes to hardware — remember when Intel messed up the division algorithm inside their chips?

[tgma]

> remember when Intel messed up the division algorithm inside their chips?

Yea, and what do you remember about that? Pretty much no normal person noticed before someone constructed a specific test case.

You know what I also remember? Meltdown, Spectre, which were indeed worked around by software tweaks in OS and compilers. There have been dozens of other microcode patches, etc. You are being too lenient in your assessment of the crap that hardware engineers ship. There's just a lot more of software out there with errors that are in your face, so you tend to notice them immediately. I give you one thing though: precisely because the remediation cost of a software defect is less than a hardware defect once shipped, people are not as worried about validation in most use cases. That's not inherently a mistake though, just intuitive cost-benefit assessment.

[Joker_vD]

> Yea, and what do you remember about that?

That it was a rather big PR disaster, and it cost Intel lots of money to recall the chips. Such things don't really happen with software; software is expected to have bugs, some of them maybe even catastrophic, but that's just the way the things are.

> because the remediation cost of a software defect is less than a hardware defect once shipped, people are not as worried about validation in most use cases. That's not inherently a mistake though, just intuitive cost-benefit assessment.

Yes, and also, the users are told to simply bear up with software bugs (and some low-priority ones sometimes never get fixed). So let's jump several levels of comments up:

   > > > People who acknowledge the quality problem, but think they can deal with it by applying more AI to the output.

   > > Brute Force: if it doesn't work, you're just not using enough.

   > > What if they're right though?

   > It does not have to be brushed away as "brute force" necessarily. We can, and do, build more reliable systems out of less reliable components... Even without AI, you already have buggy compilers and buggy OSes and buggy libraries.

My problem with the last comment is that it at the same times both dismisses the software and AI's quality problem ("nah, we 'can' build reliable systems") and also acknowledges that it very noticeably exists and needs to be dealt with somehow! And really, "buy literal insurance"? Who even sells insurance against software bugs?

[tgma]

> Who even sells insurance against software bugs?

You don't buy insurance for existence of a singular "bug." You can and people very often do buy liability insurance against damage caused by software bugs. You need to buy this stuff to be able to attach indemnification to enterprise contracts.