[Art9681]

I think the problem is the way you are phrasing your argument implies the LLM is always wrong. Consider a simple prompt: "Write a hello world in Python."

Every LLM i've tested gets this correct. In my mind, it can't be both bullshit and correct.

I would argue that the amount of real bullshit returned from an LLM is correlated to the amount of bullshit you give it. Garbage in, garbage out.

In the end, its irrelevant if its a statistical engine or whatever semantics we want to use (glorified autocomplete). If it solved my problem in less time than I perceive I would have solved it without it, bullshit isn't the word I would use to describe the outputs.

In all fairness though, I do get some bullshit responses.

[lou1306]

It only gives you the statistically more likely way a conversation would evolve after one party says "Write a hello world in Python." It only happens to be the correct one.

If I ask a 5yo "42 * 21 equals...?" and the kid replies with a random number, say, "882", and gets it right, it does not mean that the kid knows what multiplication is or how it works.

[alphan0n]

ChatGPT can use a stateful python environment to do math. It isn’t confabulating the answers, it’s using a calculator.

[Terr_]

Small clarification: The LLM is still confabulating the short python expression string.

The execution of that expression is being carried out by a fundamentally separate set of algorithms, which do not have the same strengths or weaknesses.

[mercer]

I mean that's just confabulating the next token with extra steps... ime it does get those wrong sometimes. I imagine there's an extra internal step to validate the syntax there.

I'm not arguing for or against anything specifically, I just want to note that in practice I assume that to the LLM it's just a bunch of repeating prompts with the entire convo, and after outputting special 'signifier' tokens, the llm just suddenly gets a prompt that has the results of the program that was executed in an environment. for all we know various prompts were involved in setting up that environment too, but I suspect not.

[Terr_]

> In my mind, it can't be both bullshit and correct.

It's easy for bullshitters to say some true things, but it doesn't change the nature of the process that got the results. Ex:

________

Person A: "The ghost of my dead gerbil whispers unto me the secrets of the universe, and I am hearing that the local volcano will not erupt today."

Person B: "Bullshit."

[24 hours later]

Person A: "See? I was correct! I demand an apology for your unjustified comment."