[thomastjeffery]

It blows my mind how absurd the narrative around LLMs has gotten. It's incredible that this subject has been talked in circles so far that trivial realities like this aren't immediately obvious to everyone.

LLMs do not count. When you ask an LLM to count (which is something it does not do), in the end no counting has happened. No shit, Sherlock.

[polotics]

So what is your point?

Claims of context-window sizes increases do imply that information can be extracted from such windows. So this is not trivial.

The counting example is a simple case, the "how much is this insurance policy going to cover this damage" is also a calculation.

[thomastjeffery]

But the LLM never calculates anything! Why in the world would I ever expect a calculated result?

An LLM doesn't even answer a question, either: it continues a prompt. That continuation looks like an answer to you and me; but to the LLM, it contains nothing more than the most likely tokens.

I keep hearing this story that an LLM is capable of objectivity, and that it's just relatively bad at it. The LLM literally never does objectivity. It can't be bad at a thing it never does.

Every time someone calls this sort of interaction a "limitation", they are only obfuscating the narrative with a useless anthropomorphization. The LLM is not a person. It is not a mind. It does not perform objective thought. It is a statistical model that provides the most likely text. No more, no less.