[nyxtom]

For every interesting problem AI solves there are a long tail of really dumb things that AI performs that humans would never do. Some days I am in awe of one-shot magic eight-ball output and other days I'm so frustrated by the sheer stupidity of what it produces. It remains to be seen whether that long tail of stupidity can ever be resolved in the current form of LLMs.

[cout]

Are you sure you aren't underestimating human capacity for stupidity?

[HighGoldstein]

At some point I was thinking that maybe I am too hard on the AI and that humans routinely produce exceptionally stupid code, however after a while I've realized that this is only partially true. AI produces a class of mistakes that humans would almost certainly not make because creating even the context of the mistake would require a level of skill that would preclude such mistakes. It's like if you took a mid/senior engineer and randomly lobotomized them mid-task.

[nyxtom]

Exactly, when you hire a junior engineer the distribution of code quality is fairly well known. By the time an engineer becomes senior, you can generally expect similar level of quality across any given task. Whereas with AI, sometimes the output is senior and in the middle of a task you'll get unpredictable low quality output. This makes the system both frustrating and unreliable. Now apply that to other domains like self-driving vehicles, where perhaps 80% of the time on a generally stress free freeway it does fine, and then randomly it may decide mid drive to slam on the brakes because of a random variance in the sensor stack.

[glaslong]

We might be undervaluing consistency. You can plan for reliably stupid. Harder to rely on Often Smart, but Unpredictably Stupid.

[hodgehog11]

I supervise quite a few Masters students. In my particular setting, believe me, LLM stupid for the top three chatbots is easier to work with than real human stupid now. We passed that threshold earlier this year.

[cousinbryce]

It’s in the training data!

[andrei_says_]

It’s like a constant game of two truths and a lie.

[briandw]

You are describing the Jagged Frontier. https://mitsloan.mit.edu/sites/default/files/2023-10/SSRN-id...

[daemin]

Very much like gambling, you can hit the jackpot, or just have continuous near (and far) misses.

[epistasis]

A million monkeys at typewriters, but with momentary runs of extreme luck/brilliance.

[Nevermark]

It worked for humans. It took a lot of us, but eventually we accepted zero as a number. Then negative numbers. Then "imaginary numbers" as a useful trick, and then as meaningful.

In our case, hundreds of millions, but we got there.