[laichzeit0]

You could push the analogy even further and run the thought experiment where every forward pass through an LLM could in principle be done on pen and paper, distributed throughout all humanity. Sure it would take a long time, but the output would be exactly the same. We’ve just shifted the implementation from GPU to scribbling things down on paper. If you want to assert that LLMs are “conscious” then you would have to likewise say this pen-and-paper implementation is conscious unless you want to say a certain clock-speed is a necessary condition for consciousness.

[jhbadger]

When we get complete neuronal connection maps (which we are getting close to for mice and humans will be done within a decade or two), we could in principle simulate a brain on a computer or on paper too. Unless you assert something magical like a "soul", these connections are what determine human consciousness. It is one thing to argue that LLMs don't resemble brains and if they could be "conscious" they wouldn't be conscious in the sense we are, but asserting that anything understandable can't be conscious won't age well.

[kbelder]

While I think you're right in principle, there's a lot of reason to think the structure of a brain is more complicated than just the connection map of the neural network. There is a lot of complicated behavior inside each neuron that we don't fully understand yet. They aren't just logic gates.

But, of course, that's just physics. It's not magic, so your point stands.

[threethirtytwo]

We know the brain can be modeled by math (and therefore thought can be written down on paper).

We know because we have mathematical models for atoms. And we know the brain is made out of atoms therefore the brain is simply a mathematical model of interconnected atoms that form a specific structure called the brain.

Thusevery facet of macro (keyword) reality should be able to be written on paper and calculated. That goes for everything… from the emotions you feel to the internal forward pass of an LLM.

[jubilanti]

This is literally Searle's Chinese Room thought experiment.

https://en.wikipedia.org/wiki/Chinese_room

[laichzeit0]

I feel like Searle could have taken the argument further down to pen-and-paper because people will somehow think if you can just make the neural network big and fast enough then “mind” and “consciousness” will somehow emerge from the symbol manipulation being done, but if you were to write it down on paper then it wouldn’t. So yeah, if you think that consciousness can arise from computation, then you’re forced to admit it can arise through doing math on paper.

[vereis]

the problem with this is I'd strongly argue that you could do this pen and paper process with the human brain and our consciousness too; we just lack enough understanding to put pen to paper in that case

the notion of consciousness being something an experience that other animals/humans share is entirely faith based.

the only person with evidence of ones consciousness is the person claiming they're conscious.

[lelanthran]

> the problem with this is I'd strongly argue that you could do this pen and paper process with the human brain and our consciousness too; we just lack enough understanding to put pen to paper in that case.

You're basing your premise on a lack of understanding[1], the GP's premise is based on an exact understanding[2].

You don't see the difference between your premise and the GP'S premise?

-----------------

[1] "We don't know how brains actually come up with the things they come up with, like consciousness"; IOW, we don't know what the secret ingredient is, or even if there is one.

[2] "We can mechanically do the following steps using 18th-century tech and come up with the same result as the LLM"; IOW, every ingredient in here is known to us.

[threethirtytwo]

We know the brain is made up of atoms and we know how to model atoms. So we do know for a fact that the brain can be modeled mathematically and we do know that human thought can be written down symbolically as an algorithm on paper even though we don’t explicitly know the exact formulation of said algorithm… That is fact.

The blue brain project has already modeled the hippocampus and cortex of the rat brain uses advanced imaging and simulations in super computers. So if it can be written down as memory on disk it can be done on paper as well.

The rat brain is simply a smaller and structurally different neural network then the human counterpart so the jump from the blue brain project to human brains is simply a scaling issue.

But from this you should begin to see the analysis from another level. Even though we have parts of the rat brain emulated computationally we still do not know if the rat is conscious. We don’t understand the rat brain in the SAME way we do not understand the LLM.

What people are getting at is the projection of this logic to things that don’t exist yet but can exist. When the blue brain project scales to the human brain we will hit the same problem with the human brain because it’s just a scaling issue.

To sum it up. We CAN already model biological brains as mathematical equations as we do LLMs. And for both cases we still cannot fully understand or characterize the nature of both because the sheer complexity of the models are too high.

[lelanthran]

> We know the brain is made up of atoms and we know how to model atoms.

Incorrect. There's still a lot we don't know about atoms. We can (sort of) model them, but not with the degree of accuracy you appear to think we have.

I mean, it's only recently that we discovered surprising changes in the properties of quarks, gluons and nucleons in relation to each other!

So, yeah, the following foundation for your argument:

> So we do know for a fact that the brain can be modeled mathematically

Is untrue. We can't do that, we have never done that.

> The blue brain project has already modeled the hippocampus and cortex of the rat brain uses advanced imaging and simulations in super computers.

They've got something, but they don't know how close or how far away they are from accuracy to the real thing.

We've almost always had a model of the human brain; first our model was simple (it has four or five parts), then we learned more and our model expanded to include actual cells (neurons, dendrites, etc), then we learned even more and our model was refined even further to include activation energies, rerouting, etc.

What makes you think we are anywhere close to the base layer when there is no more refinement to be made? Because while there is still things in brains that our outside of our knowledge (which, by definition, we don't know yet), we don't know enough about brains to make a replica of one as a mathematical model, or in silicon.

[threethirtytwo]

> Incorrect. There's still a lot we don't know about atoms. We can (sort of) model them, but not with the degree of accuracy you appear to think we have.

Not incorrect. You are misinformed and getting pedantic. Our knowledge of atoms is enough to model macro level phenomena and has spawned fields such as materials science and molecular biology. What is intractable is the computational power needed to accurately model things like the physics of protein folding. The computational power needed for that scales exponentially such that we can’t model it. That is the reality.

That being said we don’t need to model quantum level phenomena to model macro level effects like the biological mechanism of a neuron. There are simplified models that we can use as we have used in the blue brain project.

Additionally the thing we actually can’t model and don’t know about are extreme physics like black hole physics where the quantum world interacts with gravity but that is largely irrelevant to the topic at hand.

I hope this excerpt educates you a bit.

> Is untrue. We can't do that, we have never done that.

We haven’t done that just like we haven’t actually actualized the biggest number ever calculated by a computer. We know that number exists in theory but you’d be an idiot to claim it doesn’t exist as it’s foundational. For example the number a Google exists but no one has seen evidence for its existence. We know it through logic. From the blue brain project we can infer relatively confidently that the human brain can be emulated on silicon. This also follows from Turing completeness.

> They've got something, but they don't know how close or how far away they are from accuracy to the real thing.

The emulation is Quite accurate from imaging and emulation. The properties of the emulation that match in vitro and in vivo experimental data without specific parameter tuning. It is accurate as far as we know. That is about the same extent that we understand the human brain the LLM. The better question for you is how do you know it’s not accurate? You don’t. What we do know is that from measurable properties we understand that the blue brain emulation is accurate to the section of the mouse brain it emulates. This is exactly the same reasoning applied to LLMs… the tokens LLMs generate are remarkably inline with consciousness such that it is indistinguishable and thus can be speculated to actually BE conscious.

> What makes you think we are anywhere close to the base layer when there is no more refinement to be made? Because while there is still things in brains that our outside of our knowledge (which, by definition, we don't know yet), we don't know enough about brains to make a replica of one as a mathematical model, or in silicon.

Who says we need to make a replica of humans to make it conscious? We know the brain is made up thousands of evolutionary side effects orthogonal to the concept of consciousness like hunger, sleep and anger. All we need to do is replicate a sliver of the subset of human output we do consider as consciousness and that’s it.

But right now we can’t even fully define what that subset is and we can’t even understand how an LLM replicates human output.

What we do know is that the LLM replicates human output to a degree never done before indicating that it understands what is being told. From the evidence observed it is a valid speculation to consider it a form of consciousness. That is entirely different from saying AI is human. It is clearly not human but it is unclear whether or not it is conscious.

To be confidently claiming an LLM is not conscious is fundamentally misguided because it meeting most of our intuitive expectations of what consciousness is. It’s just people can’t face the reality that their own consciousness is not a form of exceptionalism.

[lelanthran]

> That being said we don’t need to model quantum level phenomena to model macro level effects like the biological mechanism of a neuron.

How can you know that? We haven't gotten sentience from the modeling we are doing at the moment so I am wondering why you are so confident that what we already know is sufficient to form an accurate model of brains as regards to consciousness.

> Who says we need to make a replica of humans to make it conscious?

Well what's the point of bringing up the modelling of a brain in conversation about mechanically replicating the brain via pen and paper? If a replica is not necessary and you have no idea if the macro model we have is sufficient, what's even the point of this thread?

My original point was that upthread poster said we can mechanically (i.e. without any thought, just repeating a process by hand) produce *exactly* what an LLM produces, while we cannot do the same for a brain (rat or human is not relevant).

I feel that even after all the content in this thread, that point remains true and factual - LLMs are not a blackbox to the extent that brains are.

[suputra]

faculty.ucr.edu/~eschwitz/SchwitzPapers/USAconscious-140721.htm https://share.google/6Qec5T0TKeJVdUBiB

In the same vein, is American Society already not conscious? The only difference is that it doesn't output a coherent stream of words that individuals can understand. It does however, act and react on its level (a nation state)