[[All: please don't post shallow-generic reactions to baity titles. Those are basically the same thing, a la https://en.wikipedia.org/wiki/Rubin_vase, and we're trying for something more substantive here.]]
Because there’s no evidentiary reason to deny the existence of my own phenomenal experience. I appear to experience, and the simplest conclusion to draw from that appearance is that I truly do experience.
They dont. They have input that runs through a invisible stochastic canyon. As long as there is previous experience the stochastic canyon never ends. If there is none or isignificant one, or it runs out of tokkens, it hallucinates and the illusion falls apart. There is no reasoning, just the invisible grand canyon of all of human experience and knowledge. PS: try to get it to retell you a clichee movie or book and you can see life near the end, how the delta of all the same movies opens up into wildly different endings.
To advance further it would need the ability to abstract away the general situation shape and pattern recognize similar situations.
When a mathematician reads a hundred-year-old math paper, they are reproducing in their head the reasoning of someone who died long ago. That is, reasoning can be written down and replicated.
If that works, I think it's fair to say that LLM's are inanimate processes that can generate real reasoning. You can tell when you read it and it makes sense.
There are likely some kinds of reasoning that can't be written down, as well as other forms of understanding, but they also don't replicate nearly as easily.
There is a streamer who plays Diablo 2 by listening to the AI advice and it is quite funny since it is pretty clear that most of the advice is an amalgamation of random, often incorrect advicem
I wonder if it is the same for programming or not, but I vibe coded an android app just to see if I can and it just works. It required a lot of "build the code and correct the errors" pushing though.
For example requested code in kotlin but received something else.
As somebody who uses Claude heavily and heavily plays D2R it’s clear he wasn’t using Claude opus…… maybe Haiku or something. Opus isn’t as brain dead as what was being displayed
It's probably helpful in this discussion to make a difference between two definitions of reasoning:
1. phenomenal reasoning, requiring consciousness and subjective experience
2. functional reasoning, transforming premises into conclusions using logic
I think you are attacking this using definition 1, whereas the article is obviously aiming at a different type of reasoning, and trying to formalize what is actually going on. It seems to be a genuine effort.
>1. phenomenal reasoning, requiring consciousness and subjective experience
I think it is incumbent upon anyone arguing that something does not posses any given property to provide a non-circular definition of what it is that they are declaring an absence of.
All of the descriptions of experiential reasoning are usually defined in terms of rephrasing of the claim "true understanding", "conscious", "aware", "knowing" all hinge on a synonymous aspect of the words that try and shift the responsibly of explanation to the next term used in a cyclic manner.
For the weaker sense of reasoning, there simply isn't any argument that it is not happening. A calculator can perform the weaker sense. The analysis of this aspect of LLMs is purely a question of how, not what.
SAT solvers are programs designed such that their execution corresponds to the reasoning process of satisfying some given constraints. But they do not contain the reasoning process, rather they embody it.
LLMs are different in that they operate on semantic features of program state. Embedding vectors assign semantic features to syntactical structures of the vector space. Operations on these syntactical structures allow the LLM to engage with semantic features of program state directly. Here the reasoning process is contained within as an object of manipulation. An LLM sensitive to the semantic features of the input sequence and that examines the logically permissible moves to derive a new sequence closer to the intended sequence (some statement to prove) just is engaging in reasoning.
I think you are missing the point of that statement
It is a claim that swimming is a word that defines a context. It is an explicit statement that the question of whether a submarine can swim has nothing to do with the capability of the submarine.
If you are asking which pigeon hole we are putting something into, the answer is "The one we put it into". This is what make the question uninteresting.
If you are asking what is it about this pigeon hole that people value and does that align with the criteria that people use to decide categorisation. That very much is an interesting and complicated question.
The statement takes meaning-as-use as a given, sure, but I think the point of the statement is that people are arguing over an uninteresting question / taking meaningless positions about a meaningless issue, rather than "hey, words are moves in a language game!". I referenced two EWDs, which provide the original statements in context (though I can't find the widely-quoted wording anywhere: I thought I remembered it being in EWD1035, but apparently not). If you think my understanding of what Dijkstra meant was wrong, could you explain further, please?
Here is Yuji Tachikawa from Japan (Mathematical Physics, String Theory, QFT) on recent progress in his own work using Fable 5 :
"I've been trying out Claude Fable recently, and last night, on a whim, I showed it my research notes about a collaborative project that's seen no progress in the past six months or so and asked for its thoughts. To my surprise, it made a non-trivial observation and essentially solved it."
"I was also surprised that it was using sympy to automatically write code and verify his own predictions."
"Fable probably seems like it properly understands string theory and has intuition too—that's my impression"
i love how anthropic puts out some bs like this every few weeks 'we saw some red bridge lights blinking in model weights when someone mentions sfo. Arent they just like us?"
Compression is the trick. Its even philosophed about if compression = intelligence.
The LLM has to compress everyy question/prompt into its system. It does so by creating rules and ways of processing data (this can lead to AGI, world models or an architecture of sub architectures like an LLM + something else). So if it should respond in a way that only reasoning people can achieve, it might be able to learn a representation of what we call reasoning.
It read enough text in itself to even know about the concept of reasoning and how you would do that.
Even if this is only stochastic, it shouldn't be so devalued as your comment comes across.
So, the best of compression is a procedural system, capable to generate from minimal input a whole world- basically just using abstract rules and metrics for reasonableness?
So- AGI is in the principles behind universe generators like NoMansSky? Is there a ultimate game- that generates this from physics up?
Yes + the capability to use this 'world' to simulate outcomes. Thats exactly what LeCun and others are trying to do with world models.
NoMansSky generator misses the complexity of encoding more than just geometry and basic quests but thats one reason why google and others show you machine learning based 'game engines' which allow you to enter these worlds. They are not doing this to replace real game engines, the do this for world research.
Compression, but also generating , using experience and previous abstractions and evaluating new state from entropy by comparing it to previous discoveries
> Do we see signs of reasoning or is it anthropomorphism?
This is the part that so many folks just don't seem to understand (probably because it's been labeled as "thinking" or "reasoning" mode, and people assume that words have meaning). It's not reasoning or thought. It's spewing tokens pretending to "think", but it's actually just generating extra "context" to help the final answer be more coherent. The model isn't doing anything it doesn't already do. It's just doing more of it to improve the quality of the final answer displayed to the user.
You're describing a process by which a 'thinking' entity uses cognition to refine a solution to a stated problem. That's a lot of words so usually we shorten this to 'reasoning'.
Do LLMs 'think'? I 'think' they do in a way. I don't really know how I think myself but I know I do and therefore I am (thanks, Descartes). I have a somewhat better grasp of the way LLMs 'think'. They do so sequentially, building a chain of descriptors which best fit the problem and the preceding descriptors. I suspect I do something not entirely dissimilar- i.e. I imagine 'worlds' which are like the current one changed in some way so they the problem I'm working on is reduced, then refine those until it is resolved - but in a massively parallel way.
Honestly, people need to get over this debate. It's pretty irrelevant in a lot of cases. When people ask "what is the model thinking?", they're really asking "what caused the model to produce this response (as opposed to a bunch of other plausible ones)?"
Whether it's thinking or word prediction or whatever you want to call it, people are trying to understand the causal chain.
It's not just a nominalistic debate though, as the people who are vocal against the idea that LLMs might "understand" or "think" also claim that because of this, they are fundamentally limited in what they can achieve, in contrast to human beings. Therefore any possibility of actual intelligence (or even superintelligence) is, according to them, just a fantasy.
The papers linked in the article discuss the mechanical operations that simulate reasoning. Intelligence is data efficiency and I don't see a strong argument that reasoning can exist if it requires a world's worth of data.
That doesn't mean that simulated reasoning isn't useful, it's wildly useful. But a thing is not its simulation.
"The King leaned over, looked and saw, yes, the Middle Ages simulated to a T, all digital, binary , and nonlinear, and there was the land of Dandelia, The Icicle Forest, the palace with the Helical Tower, the Aviary That Neighed, and the Treasury with a Hundred Eyes as well, and there was Ineffabelle herself, taking a slow, stochastic stroll through the simulated garden, and her circuits glowed red and gold as she picked simulated daisies, and hummed a simulated song."
Yes, there is an LLM feature that we have anthropomorphized as "reasoning" or "thinking", where an LLM has a scratch space where it can dump tokens that help to improve the final output.
Of course they do, how else do you think they manage to implement new features in large codebases, or to prove new theorems? But you don't even have to assume they do because of the results- you can read their chain of thought.
The article body does not presume they reason.