[nerdjon]

> When I use AI to perform X, every single time I run that AI from now until the heat death of the sun it will maybe produce Y. Forever! When it does, we don't understand why, and when it doesn't, we also don't understand why!

To make this even worse, it may even produce Y just enough times to make it seem reliable and then it is unleashed without supervision, running thousands or millions of times, wrecking havoc producing Z in a large number of places.

[ryandrake]

Exactly. Fundamentally, I want my computer's computations to be deterministic, not probabilistic. And, I don't want the results to arbitrarily change because some company 1,500 miles away from me up-and-decided to "train some new model" or whatever it is they do.

A computer program should deliver reliable, consistent output if it is consistently given the same input. If I wanted inconsistency and unreliability, I'd ask a human to do it.

[ethbr1]

The problem with this is that it runs counter to AI company valuations.

Their valuations: AI all the things

Reality: AI the minimum number of steps, surrounded by guardrails and traditional deterministic automation

But AI companies won't be worth AI money if that reality persists.

[LightBug1]

It's not arbitrary ... your precise and deterministic, multi-year, financial analysis needs to be corrected every so often for left-wing bias.

/s ffs

[cteiosanu]

> To make this even worse, it may even produce Y just enough times to make it seem reliable and then it is unleashed without supervision, running thousands or millions of times, wrecking havoc producing Z in a large number of places.

Reminds me of the famous Xerox scanning bug: https://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres...