[fnordpiglet]

No, because we are able to extrapolate from our experience. The ability to synthesize something coherent that doesn’t map directly into our training set is a major difference between human intelligence and what we call AI today.

[alexeldeib]

Isn’t there an argument we’re simply better at brain statistics and modeling than current AI? Forget architectural limitations. What is the nature of the extrapolation? How do individuals balance their experiences and determine likely outcomes?

[fnordpiglet]

Maybe! But even so there’s facilities AI lack that are more capability based than model based. For instance we demonstrate agency, we can simulate things in our mind alone, such as arriving at Maxwells Equations, or general relativity, or any number of other profound insights that aren’t based on our training data but are an extrapolation through our mind into domains we’ve no experience with and arrive at profound insights never conceived of before. Statistical models generally aren’t able to do this - they’re reflections of their training set, even if very complex ones. The human mind can create its own training set and that’s a remarkable capability.

[Der_Einzige]

The overwhelming majority of human advancements is interpolation. Extrapolation is rare and we tend to only realize something was extrapolation after the fact.

[fnordpiglet]

Some achievement are very clearly extrapolation. Galois field theory, general relativity, Greens theorem, to name a few I’m familiar with. These are the leaps of extrapolation that change the world - but the human mind has capabilities LLM simple can’t - agency, facilities at speculative simulation of approximations (dreaming and imagination), random recall memory, among others. These aren’t woowoo magic human ideas they’re measurable and identifiable capabilities that by construction LLMs can’t have - even if they can approximate it in a compelling way. That doesn’t mean we can’t build something with these and other intelligence capabilities AI of today lack or that they aren’t profoundly useful. But they literally can’t do anything but “walk” their vector space - and nothing but their vector space.

[strangescript]

"extrapolate from our experience" "synthesize something coherent"

These are non-scientific concepts. You are basically saying "humans are doing something more, but we can't really explain it".

That assumption is getting weaker by the day. Our entire existence is a single, linear, time sequence data set. Am I "extrapolating from my experience" when I decide to scratch my head? No, I got a sequential data point of an "itch" and my reward programming has learned to output "scratch".

[fnordpiglet]

Are you saying the discovery of relativity happened because Einstein was reacting to some reward / stimulus in his environment? Galois’ discoveries were a stochastic parrot regurgitating stimulus from his life?

There are known faculties humans have that LLMs especially do not, such as actual memory, the ability to simulate the world independently via the imagination and structured thought, as well as facilities we don’t really understand but AIs definitely don’t have which are the source of our fundamental agency. We are absolutely able to create thought and reasoning without direct stimulus or as a response to something in the environment - and it’s frankly bizarre a human being can believe they’ve never done something as a reaction to their internal state rather than extrinsic.

LLMs literally can not “do” anything that isn’t predicated on their training set. This means, more or less, they can only interpolate within their populated vector space. The emergent properties are astounding and they absolutely demonstrate what appears to be some form of pseudo abductive reasoning which is powerful. I think it’s probably the most important advance of computing in the last 30 years. But people have confused a remarkable capability for a human like capability, and have simultaneously missed the importance of the advance as well as inexplicably diminished the remarkable capabilities of the human mind. It’s possible with more research we will bridge the gaps, and I’m not appealing to magic of the soul here.

But the human mind has a remarkable ability to reason, synthesize, extrapolate beyond their experience, and those are all things LLMs fundamentally - from a rigorous mathematical basis - can not do and will never do alone. Any thing that bridges that will need an ensemble of AI and classical computing techniques - and maybe LLMs will be a core part of a part of something even more amazing. But we aren’t there yet and I’ve not seen a roadmap that takes us there.

[strangescript]

Or all these properties are emergent from ever increasing compute. The underlying architecture doesn't need to fundamentally change. The roadmap is increasing compute. Agency is simply never ceasing input and resulting output, both internally and externally. Do you have Agency when you sleep? I still don't know what "extrapolate beyond their experience" means. Does Einstein discover relativity if he is never taught math? More to the point, why was he so brilliant in the first place. Why could he "extrapolate beyond their experience" better than others in his field?

We want AI to hoop jump something we don't even understand ourselves. The only empirical evidence we have is as we increase compute, the results get better.

[fnordpiglet]

I would note you absolutely have agency when you sleep - in fact a lot of the purpose of dreams is to simulate situations for you to practice in safely - that’s why you’re often replaying situations you’re worried about or things you commonly experience. Also lucid dreaming is still fully immersed dreaming but you have total agency and awareness. The key to lucid dreaming is just training yourself to not give up aware of the non dream reality control while dreaming.