[akoboldfrying]

LLMs and human brains are both just mechanisms. Why would one mechanism a priori be capable of "learning abstract thought", but no others?

If it turns out that LLMs don't model human brains well enough to qualify as "learning abstract thought" the way humans do, some future technology will do so. Human brains aren't magic, special or different.

[meheleventyone]

Human brains aren’t magic in the literal sense but do have a lot of mechanisms we don’t understand.

They’re certainly special both within the individual but also as a species on this planet. There are many similar to human brains but none we know of with similar capabilities.

They’re also most obviously certainly different to LLMs both in how they work foundationally and in capability.

I definitely agree with the materialist view that we will ultimately be able to emulate the brain using computation but we’re nowhere near that yet nor should we undersell the complexity involved.

[panarky]

When someone says "AIs aren't really thinking" because AIs don't think like people do, what I hear is "Airplanes aren't really flying" because airplanes don't fly like birds do.

[meheleventyone]

This really shows how imprecise a term 'thinking' is here. In this sense any predictive probabilistic blackbox model could be termed 'thinking'. Particularly when juxtaposed against something as concrete as flight that we have modelled extremely accurately.

[lostmsu]

Yes, to a degree. A very low degree at that.

[atomicthumbs]

AIs think like a rock flies.

[nkrisc]

If I shake some dice in a cup are they thinking about what number they’ll reveal when I throw them?

[danielbln]

If I take a plane apart and throw all the parts off a cliff, will they achieve sustained flight?

If I throw some braincells into a cup alongside the dice, will they think about the outcome anymore than the dice alone?

[kortex]

https://en.wikipedia.org/wiki/Emergence

[shawabawa3]

that depends, if you explain the rules of the game you're playing and give the dice a goal to win the game, do they adjust the numbers they reveal according to the rules of the game?

If so, yes, they're thinking

[brendoelfrendo]

As someone who plays a lot of Dungeons and Dragons, it sure feels like the dice are thinking sometimes.

[nkrisc]

The rules of the game are to reveal two independent numbers in the range [1,6].

[jibal]

That's a fallacy of denial of the antecedent. You are inferring from the fact that airplanes really fly that AIs really think, but it's not a logically valid inference.

[jibal]

"Observing a common (potential) failure mode"

That's not what we have here.

"It is only a fallacy if you "P, therefore C" which GP is not (at least to my eye) doing."

Some people are willfully blind.

[fc417fc802]

Observing a common (potential) failure mode is not equivalent to asserting a logical inference. It is only a fallacy if you "P, therefore C" which GP is not (at least to my eye) doing.

[hackernewds]

Yeah at that point, just arguing semantics

[staticman2]

Whenever someone paraphrases a folksy aphorism about airplanes and birds or fish and submarines I suppose I'm meant to rebut with folksy aphorisms like:

"A.I. and humans are as different as chalk and cheese."

As aphorisms are a good way to think about this topic?

[DoctorOetker]

I agree we shouldn't undersell or underestimate the complexity involved, but when LLM's start contributing significant ideas to scientists and mathematicians, its time to recognize that whatever tricks are used in biology (humans, octopuses, ...) may still be of interest and of value, but they no longer seem like the unique magical missing ingredients which were so long sought after.

From this point on its all about efficiencies:

modeling efficiency: how do we best fit the elephant, with bezier curves, rational polynomials, ...?

memory bandwidth training efficiency: when building coincidence statistics, say bigrams, is it really necessary to update the weights for all concepts? a co-occurence of 2 concepts should just increase the predicted probability for the just observed bigram and then decrease a global coefficient used to scale the predicted probabilities. I.e. observing a baobab tree + an elephant in the same image/sentence/... should not change the relative probabilities of observing french fries + milkshake versus bicycle + windmill. This indicates different architectures should be possible with much lower training costs, by only updating weights of the concepts observed in the last bigram.

and so on with all other kinds of efficiencies.

[Davidzheng]

ofc, and probably will never understand because of sheer complexity. It doesn't mean we can't replicate the output distribution through data. Probably when we do in efficient manners, the mechanisms (if they are efficient) will be learned too.

[thesz]

  > Human brains aren't magic, special or different.

DNA inside neurons uses superconductive quantum computations [1].

[1] https://www.nature.com/articles/s41598-024-62539-5

As the result, all living cells with DNA emit coherent (as in lasers) light [2]. There is a theory that this light also facilitates intercellular communication.

[2] https://www.sciencealert.com/we-emit-a-visible-light-that-va...

Chemical structures in dendrites, not even neurons, are capable to compute XOR [3] which require multilevel artificial neural network with at least 9 parameters. Some neurons in brain have hundredths of thousands of dendrites, we are now talking of millions of parameters only in single neuron's dendrites functionality.

[3] https://www.science.org/doi/10.1126/science.aax6239

So, while human brains aren't magic, special or different, they are just extremely complex.

Imagine building a computer with 85 billions of superconducting quantum computers, optically and electrically connected, each capable of performing computations of a non-negligibly complex artificial neural network.

[Kim_Bruning]

All three appear to be technically correct, but are (normally) only incidental to the operation of neurons as neurons. We know this because we can test what aspects of neurons actually lead to practical real world effects. Neurophysiology is not a particularly obscure or occult field, so there are many many papers and textbooks on the topic.(And there's a large subset you can test on yourself, besides, though I wouldn't recommend patch-clamping!)

[thesz]

  > We know this because we can test what aspects of neurons actually lead to practical real world effects.

Electric current is also quantum phenomena, but it is also very averaged in most circumstances that lead to practical real world effects.

What is wonderful here is that contemporary electronics wizardry that allowed us to have machines that mimic some of thinking, also is very concerned of the quantum-level electromagnetic effects at the transistor level.

[Kim_Bruning]

On reread, if your actual argument is that SNN are surprisingly sophisticated and powerful, and we might be underestimating how complex the brain's circuits really are, then maybe we're in violent agreement.

[danielbln]

They are extremely complex, but is that complexity required for building a thinking machine? We don't understand bird physiology enough to build a bird from scratch, but an airplane flies just the same.

[thesz]

The complexities of contemporary computers and complexities of computing-related infrastructure (consider ASML and electricity) are orders of magnitudes higher than what was needed for first computers. The difference? We have something that mimics some aspects of (human) thinking.

How complex our everything computing-related should be to mimic thinking (of humans) little more closely?

[sublinear]

Are we not just getting lost in semantics when we say "fly"? An airplane does not at all perform the same behavior as a bird. Do we say that boats or submarines "swim"?

Planes and boats disrupt the environments they move through and air and sea freight are massive contributors to pollution.

[IAmBroom]

You seem to have really gone off the rails midway through that post...

[d-lisp]

(Motors and human brains are both just mechanisms, the reason one is a priori capable of learning abstract thought and not the other ?)

While I agree to some extent with the materialistic conception, the brain is not an isolated mechanism, but rather the element of a system which itself isn't isolated from the experience of being a body in a world interacting with different systems to form super systems.

The brain must be a very efficient mechanism, because it doesn't need to ingest the whole textual production of the human world in order to know how to write masterpieces (music, litterature, films, software, theorems etc...). Instead the brain learns to be this very efficient mechanism with (as a starting process) feeling its own body sh*t on itself during a long part of its childhood.

I can teach someone to become really good at producing fine and efficient software, but on the contrary I can only observe everyday that my LLM of choice keeps being stupid even when I explain it how it fails. ("You're perfectly right !").

It is true that there's nothing magical about the brain, but I am pretty sure it must be stronger tech than a probabilistic/statistical next word guesser (otherwise there would be much more consensus about the usability of LLMs I think).

[RayVR]

I'm not arguing that human brains are magic. the current AI models will probably teach us more about what we didn't know about intelligence than anything else.

[jpkw]

Right, I'm just going to teach my dog to do my job then and get free money as my brain is no more magic, special or different to theirs!

[nephihaha]

There isn't anything else around quite like a human brain that we know of, so yes, I'd say they're special and different.

Animals and computers come close in some ways but aren't quite there.

[littlestymaar]

> LLMs and human brains are both just mechanisms. Why would one mechanism a priori be capable of "learning abstract thought", but no others?

“Internal combustion engines and human brains are both just mechanisms. Why would one mechanism a priori be capable of "learning abstract thought", but no others?”

The question isn't about what an hypothetical mechanism can do or not, it's about whether the concrete mechanism we built does or not. And this one doesn't.

[ben_w]

The general argument you make is correct, but you conclusion "And this one doesn't." is as yet uncertain.

I will absolutely say that all ML methods known are literally too stupid to live, as in no living thing can get away with making so many mistakes before it's learned anything, but that's the rate of change of performance with respect to examples rather than what it learns by the time training is finished.

What is "abstract thought"? Is that even the same between any two humans who use that word to describe their own inner processes? Because "imagination"/"visualise" certainly isn't.

[rob74]

> no living thing can get away with making so many mistakes before it's learned anything

If you consider that LLMs have already "learned" more than any one human in this world is able to learn, and still make those mistakes, that suggests there may be something wrong with this approach...

[ben_w]

Not so: "Per example" is not "per wall clock".

To a limited degree, they can compensate for being such slow learners (by example) due to the transistors doing this learning being faster (by the wall clock) than biological synapses to the same degree to which you walk faster than continental drift. (Not a metaphor, it really is that scale difference).

However, this doesn't work on all domains. When there's not enough training data, when self-play isn't enough… well, this is why we don't have level-5 self-driving cars, just a whole bunch of anecdotes about various different self-driving cars that work for some people and don't work for other people: it didn't generalise, the edge cases are too many and it's too slow to learn from them.

So, are LLMs bad at… I dunno, making sure that all the references they use genuinely support the conclusions they make before declaring their task is complete, I think that's still a current failure mode… specifically because they're fundamentally different to us*, or because they are really slow learners?

* They *definitely are* fundamentally different to us, but is this causally why they make this kind of error?

[quantummagic]

But humans do the same thing. How many eons did we make the mistake of attributing everything to God's will, without a scientific thought in our heads? It's really easy to be wrong, when the consequences don't lead to your death, or are actually beneficial. The thinking machines are still babies, whose ideas aren't honed by personal experience; but that will come, in one form or another.

[littlestymaar]

> The thinking machines are still babies, whose ideas aren't honed by personal experience; but that will come, in one form or another.

Some machines, maybe. But attention-based LLMs aren't these machines.

[quantummagic]

I'm not sure. If you see what they're doing with feedback already in code generation. The LLM makes a "hallucination", generates the wrong idea, then tests its code only to find out it doesn't compile. And goes on to change its idea, and try again.

[littlestymaar]

> but that's the rate of change of performance with respect to examples rather than what it learns by the time training is finished.

It's not just that. The problem of “deep learning” is that we use the word “learning” for something that really has no similarity with actual learning: it's not just that it converges way too slowly, it's also that it just seeks to minimize the predicted loss for every samples during training, but that's no how humans learn. If you feed it enough flat-earther content, as well a physics books, an LLM will happily tells you that the earth is flat, and explain you with lots of physics why it cannot be flat. It simply learned both “facts” during training and then spit it out during inference.

A human will learn one or the other first, and once the initial learning is made, it will disregards all the evidence of the contrary, until maybe at some point it doesn't and switches side entirely.

LLMs don't have an inner representation of the world and as such they don't have an opinion about the world.

The humans can't see the reality for itself, but they at least know it exists and they are constantly struggling to understand it. The LLM, by nature, is indifferent to the world.

[ben_w]

> If you feed it enough flat-earther content, as well a physics books, an LLM will happily tells you that the earth is flat, and explain you with lots of physics why it cannot be flat.

This is a terrible example, because it's what humans do as well. See religious, or indeed military, indoctrination. All propaganda is as effective as it is, because the same message keeps getting hammered in.

And not just that, common misconceptions abound everywhere and not just conspiracy theories, religion, and politics. My dad absolutely insisted that the water draining in toilets or sinks are meaningfully influenced by the Coriolis effect, used an example of one time he went to the equator and saw a demonstration of this on both sides of the equator. University education and lifetime career in STEM, should have been able to figure out from first principles why the Coriolis effect is exactly zero on the equator itself, didn't.

> A human will learn one or the other first, and once the initial learning is made, it will disregards all the evidence of the contrary, until maybe at some point it doesn't and switches side entirely.

We don't have any way to know what a human would do if they could read the entire internet, because we don't live long enough to try.

The only bet I'd make is that we'd be more competent than any AI doing the same, because we learn faster from fewer examples, but that's about it.

> LLMs don't have an inner representation of the world and as such they don't have an opinion about the world.

There is evidence that they do have some inner representation of the world, e.g.:

• https://arxiv.org/abs/2506.02996

• https://arxiv.org/abs/2404.18202

[littlestymaar]

> This is a terrible example, because it's what humans do as well. See religious, or indeed military, indoctrination. All propaganda is as effective as it is, because the same message keeps getting hammered in.

You completely misread my point.

The key thing with humans isn't that they cannot believe in bullshit. They can definitely do. But we don't usually believe in both the bullshit and in the fact the BS is actually BS. We have opinions on the BS. And we, as a species, routinely die or kill for these opinions, by the way. LLM don't care about anything.

[ben_w]

> But we don't usually believe in both the bullshit and in the fact the the BS is actually BS.

I can't parse what you mean by this.

> LLM don't care about anything.

"Care" is ill-defined. LLMs are functions that have local optima (the outputs); those functions are trained to approximate other functions (e.g. RLHF) that optimise other things that can be described with functions (what humans care about). It's a game of telephone, like how Leonard Nimoy was approximating what the script writers were imagining Spock to be like when given the goal of "logical and unemotional alien" (ditto Brent Spiner, Data, "logical and unemotional android"), and yet humans are bad at writing such characters: https://tvtropes.org/pmwiki/pmwiki.php/Main/StrawVulcan

But rather more importantly in this discussion, I don't know what you care about when you're criticising AI for not caring, especially in this context. How, *mechanistically*, does "caring" matter to "learning abstract thought", and the question of how closely LLMs do or don't manage it relative to humans?

I mean, in a sense, I could see why someone might argue the exact opposite, that LLMs (as opposed to VLMs or anything embodied in a robot, or even pure-text agents trained on how tools act in response to the tokens emitted) *only* have abstract "thought", in so far as it's all book-learned knowledge.

[arowthway]

For some unexplainable reason your subjective experience happens to be localized in your brain. Sounds pretty special to me.

[mapontosevenths]

There is nothing special about that either. LLM's also have self awareness/introspection, or at least a some version of it.

https://www.anthropic.com/research/introspection

Its hard to tell sometimes because we specifically train them to believe they don't.

[arowthway]

Thanks for the link, I haven't seen this before and it's interesting.

I don't think the version of self awareness they demonstrated is synonymous with subjective experience. But same thing can be said about any human other then me.

Damn, just let me believe all brains are magical or I'll fall into solipsism.

[jibal]

Thermometers and human brains are both mechanisms. Why would one be capable of measuring temperature and other capable of learning abstract thought?

> If it turns out that LLMs don't model human brains well enough to qualify as "learning abstract thought" the way humans do, some future technology will do so. Human brains aren't magic, special or different.

Google "strawman".