[kqr]

No but that does happen a lot of the time. The difference is that we can choose to engage the deductive engine to verify what the parrot says. Sometimes it's easier to do so (what's 17×9?) and sometimes it's harder (a ball and a bat costs $1.1 and the bat costs $1 more than the ball, what does the ball cost?)

[vidarh]

You can ask ChatGPT to show it's working too. It's likely, for many things, (as it is with humans a lot of the time) that the way it does it when it walks you through the process is entirely different to how it does it when it just spits out an answer, but it does work. E.g. I asked it a lot of questions about the effects of artificial gravity using a rotating habitat a few days ago, and had it elaborate a lot of the answers, and it was able to set out the calculations step by step. It wasn't perfect - it made the occasional mistakes - but it also corrected them when I asked follow-up questions about inconsistencies etc. the same way a human would.

(Funnily enough, while both your example questions are easy enough, I feel I was marginally slower on your "easy" question than your "harder" one)

[kqr]

Maybe we're getting close to what makes me doubt the utility of LLMs. Much like humans, they are quick to employ System 1 instead of System 2. Unlike humans, their System 1 is so well trained it has a response for almost everything, so it doesn't have a useful heuristic for engaging System 2.

[vidarh]

For me the utility is here today. Even if I have to carefully probe and check the responses there are still plenty of things where it's worth it to me to use it right now.