[eschaton]

Your phrasing betrays your anthropomorphization of the LLM:

> If an LLM can write a decent-ish business plan,

An LLM does not write anything in the way a person does, by coming up with what they want to say and then developing supporting arguments. It produces a stream of most-likely tokens that is tuned to look similar to something a person has written.

This is why it’s worthless to “ask” an LLM “its opinion.” It has no opinion, just a multidimensional sea of interconnected token probabilities, and has no capacity to engage in any form of analysis or consideration.

Ed Zitron is right. Ceterum censeo, LLMs esse delenda.

[ashdksnndck]

Do you say similar stuff when someone talks about the motivations of a character in fiction? Do we have to precede every comment with “I’m anthropomorphizing the LLM as a convenient shorthand when describing the behavior it is modeling”? That’s going to get old.

[eschaton]

If it helps you avoid the errors inherent in anthropomorphizing an LLM, then yes, you should be saying it. Right now, way too many people are extremely sloppy in not just their language but in their thinking around LLMs, both what they are and what they’re capable of.

The difference between that and discussing character motivations in fiction is that in fact a good author writing good characters will actually attribute motivations, struggles, background, and an inner life to their characters in order for their behavior in a story to make sense. That’s why bad writing is described as “lazy” and “formulaic,” characters are doing things because the author wants them to, not because the author has modeled them as independent actors with motivation.

[ashdksnndck]

There is already research in the literature showing that LLMs have neurons that model the gender [1], personality [2], ideology [3], and historic era [4] of the author. There’s also evidence that they model the distinction between the beliefs of the author and other characters, which has been summarized as “theory of mind” [5]. And we have only scratched the surface, with most research using small open-weight models that lag behind frontier model capabilities.

[1] Z. Yu & S. Ananiadou, “Understanding and Mitigating Gender Bias in LLMs via Interpretable Neuron Editing,” arXiv:2501.14457 (2025).

[2] J. Deng et al., “Neuron-based Personality Trait Induction in Large Language Models,” arXiv:2410.12327 (2024).

[3] J. Kim, J. Evans & A. Schein, “Linear Representations of Political Perspective Emerge in Large Language Models,” arXiv:2503.02080 (2025).

[4] W. Gurnee & M. Tegmark, “Language Models Represent Space and Time,” arXiv:2310.02207 (2023).

[5] C. Hardy, “A Sparse ToM Circuit in Gemma-2-2B,” https://xtian.ai/pages/document.pdf

[wonnage]

I don't get it, how is analysis of fictional characters relevant? Nobody is committing a logical error, fictional humans can have fictional motivations and we can talk about them. I think it's still very clear that AI "motivations" and "reasoning" are not real in any human-centric definition of the terms (see recent Apple paper), hence anthropomorphizing is an error

[Terr_]

> Do you say similar stuff when someone talks about the motivations of a character in fiction?

Depends, are we faced with the same problem where a disturbingly-large portion of people don't know the character is fictional, and/or make decisions as if it were real?

If that's still happening, then yes, keeping our unconscious assumptions in check is important.

[kelseyfrog]

I'm coining "fauxthropomorphize" as a neologism to prefix every statement about LLMs and to get the "But you're anthropomorphizing LLMs"-crowd off our collective backs. One can then just start statements like such "Fauxthropomorphizing: <the statement>".

Fauxthropomorphism

/ˈfoʊ-θrə-pə-ˌmɔːr-fɪz-əm/ (noun)

Definition:

The deliberate use of anthropomorphic language to describe non-sentient systems (such as AI models), while explicitly disclaiming belief in their consciousness, agency, or subjective experience. A stylistic or rhetorical shortcut, not an ontological claim.

Etymology:

Blend of faux (French for "false") + anthropomorphism (from Greek anthropos, "human" + morphē, "form").

Lit. “False-human-form-ism.”

[antithesizer]

Your phrasing betrays your anthropomorphization of the insufferable pedant.

[Wowfunhappy]

If the output of a stream of most-likely tokens can result in a decent-ish business plan, why shouldn't the output of a stream of most-likely tokens result in a decent-ish analysis of a business plan, or of two competing ideas?

[eschaton]

It can result in something that looks like/reads as a decent-ish business plan, or an analysis of one or two. But that doesn’t make it such because despite outward appearances no amount of planning, analysis, or comparative analysis actually took place prior to or concurrent with the generation of the tokens.

That’s the fundamental problem with anthropomorphizing LLMs: Giving their output more weight than it deserves.

[danielmarkbruce]

This idea that humans are so structured in their thinking is ridiculous.

[eschaton]

It’s a whole lot less ridiculous—especially when discussed in a context where there’s an assumption that analysis is taking place—than attributing any sort of “thought” to LLMs at all.

[eschaton]

Also, if a human is writing a business plan or something that claims to be a comparative analysis of two plans but is just writing whatever comes to mind without analysis, the result shouldn’t actually be taken any more seriously than the output of an LLM. We even have a very apt term for writing and speaking like that: “Bullshitting.”

[danielmarkbruce]

"thought"... most of my "thinking" is done in language. The various intermediate steps of the latest reasoning models show something similar.