[onion2k]

If your tool regularly lies, gaslights and produces wrong results, that's a tooling issue.

It's a human issue if you don't recognise that the code it's generated is wrong. That will never change no matter how good the tooling gets.

[kiba]

The tooling is the issue because humans designed the tooling wrong. It's a chatbot interface fined tuned to sycophancy. That's not a coincidence.

[drw85]

Would it be a human issue, if you type something into a calculator and the calculated result is wrong?

Would anyone use a calculator confidently, if the result was randomly generated?

[allarm]

Why do you think this analogy is even remotely correct? It’s well-known that LLMs produce non-deterministic results. It’s also well-known that they hallucinate. To make it even clearer, all the top LLM players make sure to remind everyone of that behavior. If calculators had similar effects and warnings, it would have been a valid analogy. Instead, you're comparing apples and oranges.

[drw85]

I think it's a valid analogy in some contexts. Like when talking to a person that is not aware of non-determinism and hallucinations. Which happens on this website very frequently.

Many people here tell you to use AI like you use a calculator. With minimal or no oversight, with full access to production systems, etc.

To let a non deterministic tool communicate on your behalf, or give it access to critical systems is evidence enough that a good number of people are not aware of these facts.

[Hamuko]

Isn't part of the problem that these tools are advertised as allowing non-coders to code? How are you gonna recognise that the code is wrong when you don't know how to code and the product is telling you that you don't even need to?