[RobRivera]

Actually this is the crux and the nuance which makes discussing LLM specifics a pain in the general space.

If you built an LLM exclusively on the writings and letters of John Steinbeck, you could NOT tell the LLM to solve an integral for you amd expect it to be right.

Instead what you will receive is a text that follows a statistically derived most likely (in accordance to the perplexity tuning) response to such a question.

[netdevphoenix]

> If you built an LLM exclusively on the writings and letters of John Steinbeck, you could NOT tell the LLM to solve an integral for you amd expect it to be right.

Isn't this obvious? There is not enough latent knowledge of math there to enable current LLMs to approximate anything resembling an integral.

[RobRivera]

Its obvious to me.

Its obvious to you.

It isnt obvious to the person I am responding to, and it isnt obvious to majority of individuals I speak with on the matter (which is why AI, personally, is in the bucket of religion amd politics for polite conversation to simply avoid)

[kenjackson]

Wait -- I'm fairly certain this is obvious to the person you were responding to. It may not be obvious to a lay person (who may not even know LLMs are trained at all). But I think this is obvious to almost all people with even a small understanding of LLMs.

[RobRivera]

I'm actually pretty convinced they're a troll or at the very least a high confrontation participant who is quick to move goal posts, ignore entire chains of logic, engage in ad hominim attacks of other posters, and is bringing zero novel insight anywhere in this thread

[simianwords]

one of the posters said it can't even reason through chess, i ran the actual benchmark, spent money and actually proved that it can beat a 1000 elo chess engine.

https://news.ycombinator.com/item?id=47316787

does this show i'm a troll? throughout this thread there has been misinformation that i have been dispelling.

what you are doing is ad hominem.

here's another post where i ran the prompt that the person asked which would apparently show that LLM's can't reason

https://news.ycombinator.com/item?id=47316855

have you considered that you might be misinformed so what i say might look like trolling?

[RobRivera]

What do you do for work?

[simianwords]

It’s obvious to me. What point are you trying to make? It’s not religion it’s falsifiable easily.

LLMs can reason about integrals as well as in a literature context. You suggested that if it’s not trained on literature then it can’t reason about it. But why does that matter?

[Talanes]

Now what if we ask the LLM to write about social media? Do you think the output would be similar to what you'd get if we had a time machine to bring the actual man back and have him form his own thoughts firsthand?

[bigfishrunning]

It may be stylistically similar, but it's impossible to predict what the content would be.

[simianwords]

>If you built an LLM exclusively on the writings and letters of John Steinbeck, you could NOT tell the LLM to solve an integral for you amd expect it to be right.

this shows that you have very less idea on how llm's work.

LLM that is trained only on john steinbeck will not work at all. it simply does not have the generalised reasoning ability. it necessarily needs inputs from every source possible including programming and maths.

You have completely ignored that LLMs have _generalised_ reasoning ability that it derives from disparate sources.

[bigfishrunning]

LLMs have the ability to convince you that they've "reasoned". sometimes, an application will loop the output of an LLM to its input to provide a "chain of reasoning"

This is not the same thing as reasoning.

LLMs are pattern matchers. If you trained an llm only to map some input to the output of John Steinbeck, then by golly that's what it'll be able to do. If you give it some input that isn't suitably like any of the input you gave it during training, then you'll get some unpredictable nonsense as output.

[simianwords]

this is outdated stuff from 3 years ago.

> If you trained an llm only to map some input to the output of John Steinbeck

this is literally not possible because the llm does not get generalised reasoning ability. this is not a useful hypothetical because such an llm will simply not work. why do you think you have never seen a domain specific model ever?

if you wanted to falsify this claim: "llm's cant reason" how would one do that? can you come up with some examples that shows that it can't reason? what if we come up with a new board game with some rules and see if it can beat a human at it. just feed the rules of the game to it and nothing else.

here is gpt-5.4 solving never before seen mathematics problems: https://epochai.substack.com/p/gpt-54-set-a-new-record-on-fr...

you could again say its just pattern matching but then i would argue that its the same thing we are doing.

[bigfishrunning]

Domain specific LLM's absolutely exist, don't assume i've never seen one. You seem very misinformed on what is "literally not possible".

https://www.ibm.com/think/topics/domain-specific-llm

[simianwords]

there are close to zero domain specific models that beat frontier SOTA models even in their own domain. (other than few edge cases like token extraction)

why do you think that's the case? lets start from here.

the real answer is that you get benefits from having data from many sources that add up expontentially for intelligence.

> LLMs are pattern matchers

but lets try to falsify this. can you come up with a prompt that clearly shows that LLM's can't reason?

[RobRivera]

What do you do for work?