[lou1306]

It only gives you the statistically more likely way a conversation would evolve after one party says "Write a hello world in Python." It only happens to be the correct one.

If I ask a 5yo "42 * 21 equals...?" and the kid replies with a random number, say, "882", and gets it right, it does not mean that the kid knows what multiplication is or how it works.

[alphan0n]

ChatGPT can use a stateful python environment to do math. It isn’t confabulating the answers, it’s using a calculator.

[Terr_]

Small clarification: The LLM is still confabulating the short python expression string.

The execution of that expression is being carried out by a fundamentally separate set of algorithms, which do not have the same strengths or weaknesses.

[mercer]

I mean that's just confabulating the next token with extra steps... ime it does get those wrong sometimes. I imagine there's an extra internal step to validate the syntax there.

I'm not arguing for or against anything specifically, I just want to note that in practice I assume that to the LLM it's just a bunch of repeating prompts with the entire convo, and after outputting special 'signifier' tokens, the llm just suddenly gets a prompt that has the results of the program that was executed in an environment. for all we know various prompts were involved in setting up that environment too, but I suspect not.