[Terr_]

> I am not sure what you mean by LLM when you say they are professional bullshitter.

Not parent-poster, but an LLM is a tool for extending a document by choosing whatever statistically-seems-right based on other documents, and it does so with no consideration of worldly facts and no modeling of logical prepositions or contradictions. (Which also relates to math problems.) If it has been fed on documents with logic puzzles and prior tests, it may give plausible answers, but tweaking the test to avoid the pattern-marching can still reveal that it was a sham.

The word "bullshit" is appropriate because human bullshitter is someone who picks whatever "seems right" with no particular relation to facts or logical consistency. It just doesn't matter to them. Meanwhile, a "liar" can actually have a harder job, since they must track what is/isn't true and craft a story that is as internally-consistent as possible.

Adding more parts around and LLM won't change that: Even if you add some external sensors, a calculator, a SAT solver, etc. to create a document with facts in it, once you ask the LLM to make the document bigger, it's going to be bullshitting the additions.

[Art9681]

I think the problem is the way you are phrasing your argument implies the LLM is always wrong. Consider a simple prompt: "Write a hello world in Python."

Every LLM i've tested gets this correct. In my mind, it can't be both bullshit and correct.

I would argue that the amount of real bullshit returned from an LLM is correlated to the amount of bullshit you give it. Garbage in, garbage out.

In the end, its irrelevant if its a statistical engine or whatever semantics we want to use (glorified autocomplete). If it solved my problem in less time than I perceive I would have solved it without it, bullshit isn't the word I would use to describe the outputs.

In all fairness though, I do get some bullshit responses.

[lou1306]

It only gives you the statistically more likely way a conversation would evolve after one party says "Write a hello world in Python." It only happens to be the correct one.

If I ask a 5yo "42 * 21 equals...?" and the kid replies with a random number, say, "882", and gets it right, it does not mean that the kid knows what multiplication is or how it works.

[alphan0n]

ChatGPT can use a stateful python environment to do math. It isn’t confabulating the answers, it’s using a calculator.

[Terr_]

Small clarification: The LLM is still confabulating the short python expression string.

The execution of that expression is being carried out by a fundamentally separate set of algorithms, which do not have the same strengths or weaknesses.

[mercer]

I mean that's just confabulating the next token with extra steps... ime it does get those wrong sometimes. I imagine there's an extra internal step to validate the syntax there.

I'm not arguing for or against anything specifically, I just want to note that in practice I assume that to the LLM it's just a bunch of repeating prompts with the entire convo, and after outputting special 'signifier' tokens, the llm just suddenly gets a prompt that has the results of the program that was executed in an environment. for all we know various prompts were involved in setting up that environment too, but I suspect not.

[Terr_]

> In my mind, it can't be both bullshit and correct.

It's easy for bullshitters to say some true things, but it doesn't change the nature of the process that got the results. Ex:

________

Person A: "The ghost of my dead gerbil whispers unto me the secrets of the universe, and I am hearing that the local volcano will not erupt today."

Person B: "Bullshit."

[24 hours later]

Person A: "See? I was correct! I demand an apology for your unjustified comment."

[widdershins]

> it does so with no consideration of worldly facts

Why don't you consider its training set (usually the entire internet, basically) worldly facts? It's true that the training set can contain contradictory facts, but usually an LLM can recognize these contradictions and provide analysis of the different viewpoints. I don't see how this is much different from what humans can do with documents.

The difference is that humans can do their own experiments and observations in the real world to verify or dismiss things they read. Providing an LLM with tools can, in a limited way, allow an LLM to do the same.

Ultimately its knowledge is limited by its training set and the 'external' observations it can make, but this is true of all agents, no?

[dartos]

LLMs are trained with a data which may contain both truthful and false information.

But at inference time it’s not referring to that data at all. Some of the data is aliased and encoded in the model’s weights, but we’re not sure exactly what’s encoded.

It may very well be that vague concepts (like man, woman, animal, unhealthy) are encoded, but not details themselves.

Further, at inference time, there is no kind of “referencing” step. We’ve just seen that they can sometimes repeat text they were trained on, but sometimes they just don’t.

The LLM based systems you’re probably using do some RAG work to insert relevant information in the LLM’s context. This context still is not being referred to per se. An LLM might have a document that says the sky is red, but still insist that it’s blue (or vice versa)

So while the info an LLM may have available is limited by its training data and the RAG system around it, none of that is guaranteed at inference time.

There’s always a significant chance for the LLM to make up bullshit.

[klibertp]

> The word "bullshit" is appropriate because human bullshitter is someone who picks whatever "seems right" with no particular relation to facts or logical consistency.

Not quite true - this is true for your random bullshitter, but professional bullshitters do, in fact, care for the impression of logical consistency and do have a grip on basic facts (if only so they can handwave them more effectively). As such, LLMs are definitely not yet pros at bullshitting :)

[zaroth]

Tell me you haven’t used the latest models, without telling me you haven’t used the latest models?

They do hallucinate at times, but you’re missing a lot of real utility by claiming they are basically bullshit engines.

They can now use tools, and maintain internal consistency over long context windows (with both text and video). They can iterate fully autonomously on software development by building, testing, and bug fixing on real world problems producing usable & functioning code.

There’s a reason Microsoft is putting $80 billion dollars on the line to run LLMs. It’s not because they are full of shit!

[enragedcacti]

Meta put $45 Billion into the Metaverse... so how much virtual real estate do you own?

[bavell]

It's true, they're very convincing bullshitters ;)