[agentultra]

It's not thinking, plain and simple.

Anything it generates means nothing to the algorithm. When you read it and interpret what was generated you're experiencing something like the Barnum-Forer effect. It's sort of like reading a horoscope and believing it predicted your future.

[myrmidon]

What gives you any confidence that the way GPT4 comes up with answers is qualitatively different from humans?

Why should the emulation of human though, a result of unguided evolution, require anything more than properly wired silicon?

[agentultra]

That's highly reductive of our capacities. We are not weighted transformers that can be explained in an arxiv paper. GPT, at the end of the day, is a statistical inference model. That's it.

It's not going to wake up one day, decide it prefers eggs benny and has had enough of your idle chatter because of that sarcastic remark you made last week.

Could we simulate a plausibly realistic human brain on silicon someday? I don't know, maybe? But that's not what GPT is and we're no where near being able to do that.

You can scale up the tokens an LLM can manage and all you get is a more accurate model with more weights and transformers. It's not going to wake up one day, have feelings, religion, decide things for itself, look in a mirror and reflect on its predicament, lament the poor response it gave a user, and decide it doesn't want to live with regret and correct its mistakes.

[myrmidon]

> That's highly reductive of our capacities.

I'm not saying that GPT4 is as capable as a human-- it can not be, by design, because its architecture lacks memory/feedback paths that we have.

What I'm saying is that HOW it thinks might already be quite close in essence to how WE think.

> We are not weighted transformers that can be explained in an arxiv paper. GPT, at the end of the day, is a statistical inference model. That's it.

That is true but uninteresting-- my counterpoint is: If you concede that our brain is "simulatable", then you basically ALREADY reduced yourself to a register based VM-- the only remaining question is: what ressources (cycles/memory) are required to emulate human thought in real time, and what is the "simplest" program to achieve it (that might be something not MUCH more complicated than GPT4!).

[agentultra]

> What I'm saying is that HOW it thinks might already be quite close in essence to how WE think.

How would one be able to prove this? Nobody knows how we think, yet.

All one can say is that what GPT-4 outputs could plausible fool another human into believing another human wrote it. But that's exactly what it's designed to do, so what's interesting about that?

> If you concede that our brain is "simulatable",

It could be. Maybe. It might be that's what the universe is doing right now. Does it matter?

We're talking about writing an emulator on a Harvard-architecture computer that can fully simulate the physics and biological processes the make up a human brain. By interpreting this system in our emulator we'd be able to witness a new human being that is indistinguishable from one that isn't simulated, right?

That's not what GPT is doing. Not even close.

It turns out there's more to being human than being a register VM. Ever get punched in the face? Bleed? Fall in love? Look back on your life and decide you want to change? Write a book but never show it to anyone? Raise a child? Wonder why you dreamt about airplanes on Mars with your childhood imaginary friend? Why you hate bananas but like banana bread? Why you lie to everyone around you about how you really feel and are offended when others don't tell you the truth?

It's not so simple.

[myrmidon]

> We're talking about writing an emulator on a Harvard-architecture computer that can fully simulate the physics and biological processes the make up a human brain. By interpreting this system in our emulator we'd be able to witness a new human being that is indistinguishable from one that isn't simulated, right?

My point is: if you don't believe that there is magic pixy dust in our brains, then this would NECESSARILY be possible.

It would almost certainly be HIGHLY inefficient-- the "right way" to do AGI would be to find out which algorithmic structures are necessary for human level "performance", and implement them in a way that is suitable for your VM.

I'm arguing that GPT4 is essentially the second approach-- it lacks features for full human level performance BY DESIGN (e.g. requires pre-training, no online learning, etc.), but there is no reason to assume that the way it operates is fundamentally different from how *parts* of OUR mind work.

> It turns out there's more to being human than being a register VM. Ever get punched in the face? Bleed? Fall in love? Look back on your life and decide you want to change? Write a book but never show it to anyone? Raise a child? Wonder why you dreamt about airplanes on Mars with your childhood imaginary friend? Why you hate bananas but like banana bread? Why you lie to everyone around you about how you really feel and are offended when others don't tell you the truth?

I don not understand what you are getting at here. I consider myself a biological machine-- none of this is inconsitent with my worldview. I believe that a silicon based machine could emulate all of this if wired up properly.

PS: I often talk with people that explicitly DONT believe into the "pixy dust in our brains" (call it soul if you want), but on the other hand they strongly doubt the feasibility of AGI-- this is internally inconsistent and simply not a defensible point of view IMO.

[agentultra]

> I'm arguing that GPT4 is essentially the second approach

Ok, so then it is an algorithm that simulates a specific behaviour that produces plausibly human-level results.

My point is that this is not thinking, smart, or "general intelligence."

Let's say I write an algorithm that can also produce text. It's not an implementation of the specification for GPT-4 but something novel. It takes the exact same inputs and produces outputs that I share with you and claim is produced by GPT-4. And lo, success, you can't tell if it was produced by GTP-4 or my algorithm.

You claim it's the same thing as having GPT-4, right? If you can't tell the difference it must be the same thing.

Big deal. We can write computer programs that perform better than humans at chess, go, and now can write more text than us. We knew this was possible before we even begun on this endeavour. It's still not intelligent, conscious, smart, or anything resembling a complete human.

It's merely an algorithm that does one specific task.

> I don not understand what you are getting at here.

I've proven my point then.

There's more to the human experience than what can be simulated on a silicone chip and it doesn't have to do with hand-waving away all the complexity of reality as "magical pixie dust."

Take physical trauma. The experience of which by one human is not merely a fact. It is felt, it is reflected upon, and it is shared in the DNA of the person that experience it with their descendants. We have science investigating how trauma is shared through generations and the effects it has on our development.

You are more than a machine with inputs and outputs.

[krickkrack]

"Nobody knows how we think, yet."

Then how can you confidently say we don't think 'like' Transformers/Attention/Statistical models/etc/etc?

[robwwilliams]

I think you would love to read Mark Rowlands’ The Philosopher and the Wolf. He asks these questions and like all if us struggles with answers.

https://www.goodreads.com/book/show/8651250

[goatlover]

> If you concede that our brain is "simulatable", then you basically ALREADY reduced yourself to a register based VM-- the only remaining question is: what ressources (cycles/memory) are required to emulate human thought in real time

We haven't emulated brains yet, so we don't know. The OpenWorm project is interesting, but I don't know to what extent they've managed to faithfully recreate an accurate digital version of a nematode worm. I do know they had it driving around a robot.

Thing is that the our brains are only part of the nervous system, which extends throughout the body. So I don't know what happens if you only simulate just the brain part. Seems to me that the rest of the body kind of matters for proper functioning.

[myrmidon]

I personally believe that while interesting, projects like OpenWorm or humanbrainproject are extremely indirect and unpromising regarding AGI (or even for improving our understanding of human thinking in general).

To me, these are like building an instruction set emulator by scanning a SoC and then cobbling together a SPICE simulation of all the individual transistors-- the wrong level of abstraction and unlikely to EVER give decent performance.

People also like to point out that human neurons are diverse and hard to simulate accurately-- yeah sure, but to me that seems completely irrelevant to AGI, in the very same way that physically exact transistor modelling is irrelevant when implementing emulators.

[krickkrack]

I read this and can't help but chuckle... To say that we are nowhere being able to have AGI is quite a bold statement. It was after all only a few months ago where many people also believed we were a long way away from ChatGPT-4.

The confidence with which you think we are not weighted transformers or statistical inference models is also puzzling. How could you possibly know that? How do you know that that's not precisely what we are, or something immediately tangent to that?

Perhaps if you keep going you do get something that begins to have feeling, religion and understand that it's a self and perhaps that's precisely what happened to humans.

[agentultra]

Ah yes, the old: you can’t prove my deity doesn’t exist argument.

Puzzling that I don’t share your faith or point of view? Why?

The point is to not ascribe properties attributed to a thing we know doesn’t have them. We can teach people how ChatGPT works without getting into pseudo-philosophical babble about what consciousness is and whether humans can be accurately simulated by an LLM with enough parameters.

[myrmidon]

IMO the big blindside of your argument is that you MUST either accept that some magic happens in human brains (=> which is HARD to reconciliate with a science-inspired world-view), OR that achieving human-level cognitive performance is a pure hardware/software optimization problem.

The thing is that GPT4 already approaches human level cognitive performance in some tasks, which means you need a strong argument for WHY full human-level performance would be out of reach of gradual improvements to the current approach.

On the other hand, a very strong argument could be made that the very first artificial neural networks had the absolutely right ideas and all the improvements over the last ~40 years were just the necessary scaling/tuning for actually approaching human performance levels...

This is also where I have to recommend V Braitenbergs "Vehicles: Experiments in synthetic psychology" (from 1984!) which aged remarkably well and shaped my personal outlook on the human mind more than anything else.

[krickkrack]

What faith? I never made the claim you're attributing to me. Smug idiots like you are wrong all the time.

[TuringTest]

> What gives you any confidence that the way GPT4 comes up with answers is qualitatively different from humans?

For a start, GPT-4 doesn't include in its generation the current state of its internal knowledge used so far; any text built can only use at most the few words already generated in the current session as a kind of short-term memory.

Biological brains OTOH have a rhythm with feedback mechanisms which adapt to the situation where they're doing the thinking.

[myrmidon]

> For a start, GPT-4 doesn't include in its generation the current state of its internal knowledge used so far

Sure. But are you certain that you NEED write access to long term memory to think? Would your thinking capabilities degrade meaningfully if that was taken away?

[TuringTest]

Yes, I would say that a brain without the capacity to form new memories has degraded thinking capabilities.

[fnordpiglet]

Except for when as an expert in a field you ask it questions about that are subtle and it answers in a cogent and insightful way, and as an expert you are fully aware of that. It’s not reasonable to call that a Barnum-Forer effect. It’s perhaps not thinking (but perhaps we need to more clearly define thinking), but its not a self-deception either.