Hacker News new | ask | show | jobs
by chrisjj 17 days ago
> To effectively predict the next token it needs a good idea of what comes after the next token.

And that's all it needs. Not reasoning.

5 comments

Save us from the reasoning / sentience / consciousness / thinking semantic quicksand.

Babbage’s Analytical Engine didn’t actually analyze anything, and terminology hadn’t gotten any more clear-cut since.

+1. Inputs, processing & memory do the job.

I suspect exact and/or universal definitions for intelligence, self-awareness, 'feelings' etc will prove to be elusive, and best we'll get is systems/robots etc behaving as if possessing those qualities. With some tests to put a number on them.

Downside is that may apply to us humans too.

At some level of performance reasoning becomes the most effective method to predict the next token
Chatbot "reasoning" arises from some level of the performance of the particular user, not of the chatbot.
I’m curious what this means? I think the evidence is pretty convincing that, while brittle, there is reasoning going on (though it depends on your definition of reasoning which I’m curious what that is for you).
Isn't "reasoning" in LLMs just training it to have an internal monologue to think through problems like a human would? i.e. extra tokens.
How do you define reasoning in a measurable way?
I keep asking the same question, and I think the steelman version would be "has metacognitive patterns similar to humans"
I think the anthropomorphic view of this is dangerous in the long term as it starts the argument that anything that isn't reasoned by a human isn't reasoning at all. This just changes the argument from LLMs can reason like a human to LLMs can't reason at all while ignoring the third possibility of "LLMs can reason not like a human".

One of the biggest things I've learned after the event of LLMs is that humans definitions of intelligence/thinking/reasoning/consciousness/etc are very poorly defined. Not just across society at large, but the sciences themselves.

> I think the anthropomorphic view of this is dangerous in the long term as it starts the argument that anything that isn't reasoned by a human isn't reasoning at all.

Something like this is actually a stance in the tradition of inferentialism (see the term sapience). Though "reasoning" isn't like, turing machine computability in this space; from what I understand, it's some abstract notion of the "space of reasons". I don't really understand it, honestly.

There's some merit to this, IMO. When an LLM goes wrong, do you blame the person or the LLM? As in, would you throw said LLM in jail, and hold the LLM accountable? Not right now, at least. I'm not sure if that's what is meant by the "space of reasons", but the intuition is that 'reason' can mean a lot of different things, pragmatically speaking. Reason as a legible audit trail is one of those ways.

But that's arguably getting into the social aspect of 'reason' (important!) and not like, what STEM people traditionally think of as 'reason'.

I may be wrong but I think the word you want is anthropocentric, not anthropomorphic. Anthropomorphism is turning something that isn't human into something human, e.g. an AI, while anthropocentrism is taking human as the center and default state of things.
> possibility of "LLMs can reason not like a human"

what would be the difference between not reasoning and reasoning not like a human?

I think you're overestimating the thinking here. I ask this question very often, and the persons I ask if of practically never answer.

I ask it to highlight that most people have no clear answer to what we mean by reasoning. It's a notoriously hard term to define with an precision, and even harder to define by precision in a way that would exclude LLM's, which is usually what the people I ask it of want to do.

I don't think your steelman version really "works" for that purpose because the immediate response would be that it makes "reasoning" largely irrelevant if you define the term that restrictively.

What’s the difference?
Is regexp or compiler lookahead reasoning? No.

Mistaking chatbot lookahead as reasoning comes with being gulled by the "artificial intelligence" sales pitch.

Why are those things not reasoning?

They are narrow and limited, but still primitive forms of reasoning.