Hacker News new | ask | show | jobs
by cout 14 days ago
Are you sure you aren't underestimating human capacity for stupidity?
3 comments

At some point I was thinking that maybe I am too hard on the AI and that humans routinely produce exceptionally stupid code, however after a while I've realized that this is only partially true. AI produces a class of mistakes that humans would almost certainly not make because creating even the context of the mistake would require a level of skill that would preclude such mistakes. It's like if you took a mid/senior engineer and randomly lobotomized them mid-task.
Exactly, when you hire a junior engineer the distribution of code quality is fairly well known. By the time an engineer becomes senior, you can generally expect similar level of quality across any given task. Whereas with AI, sometimes the output is senior and in the middle of a task you'll get unpredictable low quality output. This makes the system both frustrating and unreliable. Now apply that to other domains like self-driving vehicles, where perhaps 80% of the time on a generally stress free freeway it does fine, and then randomly it may decide mid drive to slam on the brakes because of a random variance in the sensor stack.
We might be undervaluing consistency. You can plan for reliably stupid. Harder to rely on Often Smart, but Unpredictably Stupid.
I supervise quite a few Masters students. In my particular setting, believe me, LLM stupid for the top three chatbots is easier to work with than real human stupid now. We passed that threshold earlier this year.
It’s in the training data!