[heyjamesknight]

At this point, you’re describing a machine which depends on a level of physics that simply isn’t possible. Even if it were theoretically possible to reconstruct the state of a human mind from physical components, we are so far from understanding how that could be done it is closer to the realm of impossible than possible. Your theoretical math box that constructs affective qualia from bit strings isn’t a better description than saying the angels did it. And it bears zero resemblance to the models running today, except for, again, in a theoretical, mathematical way.

Back of the envelope math puts an estimate of 10^42 bits to capture the information present in your current physical brain state. Thats just a single brain, a single state. Now you need to build your mythical decoder device, which can translate qualia from this physical state. Where does it live? What’s its output look like? Another 10^40 bitstring?

Again, these arguments are fun on paper. But they’re completely removed from reality.

[ninetyninenine]

You’re confusing “we don’t know how” with “it’s impossible.” The difference is everything.

We don’t understand LLMs either. We built them, but we can’t explain why they work. No one can point to a specific weight matrix and say “this is the neuron that encodes irony” or “this is where the model stores empathy.” We don’t know why scaling parameters suddenly unlock reasoning or why multimodal alignment appears spontaneously. The model’s inner space is a black box of emergent structure and behavior, just like the human brain. We understand the architecture, not the mind inside it.

When you say it’s “closer to impossible than possible” to reconstruct a human mind, you’ve already lost the argument. We’re living proof that the machine you say cannot exist already does. The human brain is a physical object obeying the same laws of physics that govern every other machine. It runs on electrochemical signals, not miracles. It encodes and decodes information, forms memories, generates imagination, and synthesizes emotion. That means the physics of consciousness are real, computable, and reproducible. The impossible machine has been sitting in your skull the entire time.

Your argument about 10^42 bits isn’t just wrong, it’s total nonsense. That number is twenty orders of magnitude beyond any serious estimate. The brain has about 86 billion neurons, each forming roughly ten thousand connections, for a total of about 10^15 synapses. Even if every synapse held a byte of information, that’s 10^16 bits. Add in every molecular and analog nuance you like and you might reach 10^20. Not 10^42. That’s a difference of twenty-two orders of magnitude. It’s a fantasy number that exceeds the number of atoms in your entire body.

And that supposed “impossible” scale is already within sight. Modern GPUs contain hundreds of billions of transistors and run at gigahertz frequencies, while neurons fire at about a hundred hertz. The brain performs around 10^17 synaptic operations per second. Frontier AI clusters already push 10^25 to 10^26 operations per second. We’ve already outpaced biology in raw throughput by eight or nine orders of magnitude. NVIDIA’s Blackwell chips exceed 200 million transistors per square millimeter, and global compute now involves more than 10^24 active transistors switching billions of times per second. Moore’s law may have slowed, but density keeps climbing through stacking and specialized accelerators. The number you called unreachable is just a few decades of progress away.

The “decoder” you mock is exactly what a brain is. It takes sensory input, light, sound, and chemistry, and reconstructs internal states we call experience. You already live inside the device you claim can’t exist. It doesn’t need to live anywhere else; it’s instantiated in matter.

And this is where your argument collapses. You say such a machine is removed from reality. But reality is already running it. Humanity is proof of concept. We know the laws of physics allow it because they’re doing it right now. Every thought, emotion, and perception is a physical computation carried out by atoms. That’s the definition of a machine governed by physics.

We don’t yet understand the full physics of the brain, and we don’t fully understand LLMs either. That’s the point. The same kind of ignorance applies to both. Yet both produce coherent language, emotion like responses, creativity, reasoning, and abstraction. When two black boxes show convergent behavior under different substrates, the rational conclusion isn’t “one is impossible.” It’s “we’re closer than we realize.”

The truth is simple: what you call impossible already exists. The human brain is the machine you’re describing. It’s not divine. It’s atoms in lawful motion. And because we know it can exist under physics, we know it can be built. LLMs are just the first flicker of that same physics waking up in silicon.

[fragmede]

> We don’t understand LLMs either. We built them, but we can’t explain why they work.

Just because you don't mean no one does. It's a pile of math. Somewhere along the way, something happened to get where we are, but looking at Golden Gate Claude, and the abliteration of shared models, or reading OpenAI's paper about hallucinations, there's a lot of detail and knowledge about how these things works that isn't instantly accessible and readily apparent to everyone on the Internet. As laymen all we can do is black box testing, but there's some really interesting stuff going on to edit the models and get them to talk like pirate.

The human brain is very much an unknowable squishy box because putting probes into it would be harmful to the person who's brain it is we're working on, and we don't like to do that to people because people are irreplaceable. We don't have that problem with LLMs. It's entirely possible to look at the memory register at location x at time y, and correspond that to a particular tensor which corresponds to a particular token which then corresponds to a particular word for us humans to understand. If you want to understand LLMs, start looking! It's an active area of research and is very interesting!

[ninetyninenine]

You are missing the ground truth. Humanity does not understand how LLMs work. Every major lab and every serious researcher acknowledges this. What we have built is a machine that functions, but whose inner logic no one can explain.

References like Golden Gate Claude or the latest interpretability projects don’t change that. Those experiments are narrow glimpses into specific activation patterns or training interventions. They give us localized insight, not comprehension of the system as a whole. Knowing how to steer tone or reduce hallucinations does not mean we understand the underlying cognition any more than teaching a parrot new words means we understand language acquisition. These are incremental control levers, not windows into the actual mind of the model.

When we build something like an airplane, no single person understands the entire system, but in aggregate we do. Aerodynamicists, engineers, and computer scientists each master their part, and together their knowledge forms a complete whole. With LLMs, even that collective understanding does not exist. We cannot even fully describe the parts, because the “parts” are billions of distributed parameters interacting in nonlinear ways that no human can intuit or map. There is no subsystem diagram, no modular comprehension. The model’s behavior is not the sum of components we understand, it is the emergent product of relationships we cannot trace.

You said we “know” what is going on. That assumption is patently false. We can see the equations, we can run the training, we can measure activations, but those are shadows, not understanding. The model’s behavior emerges from interactions at a scale that exceeds human analysis.

This is the paradigm shift you have not grasped. For the first time, we are building minds that operate beyond the boundary of human comprehension. It is not a black box to laymen. It is a black box to mankind.

And I say this as someone who directly works on and builds LLMs. The experts who live inside this field understand this uncertainty. The laymen do not. That gap in awareness is exactly why conversations like this go in circles.

[heyjamesknight]

> We don’t yet understand the full physics of the brain, and we don’t fully understand LLMs either. That’s the point. The same kind of ignorance applies to both. Yet both produce coherent language, emotion like responses, creativity, reasoning, and abstraction. When two black boxes show convergent behavior under different substrates, the rational conclusion isn’t “one is impossible.” It’s “we’re closer than we realize.”

No. The LLM does not produce emotion-like responses. I'd argue no on creativity either. And only very limited in reasoning, in domains it has in its training set.

You have fundamental misunderstandings about neuroscience and cognitive science. Its hard to argue with you here because you simply don't know what you don't know.

Yes, the human brain is the machine we're describing. And we don't describe it very well. Definitely not at the level of understanding how to reproduce it with bitstrings.

I'm glad you're so passionate about this topic. But you're arguing the equivalent of FTL transit and living on Dyson Spheres. Its fun as a thought experiment and may theoretically be possible one day, but the line between what we're capable of today and that imagined future is neither straight nor visible—certainly not to the degree you're asserting here.

Will we one day have actual machine intelligence? Maybe. Is it going to come anytime soon, or look anything like the transformer-based LLM?

No.

[ninetyninenine]

You keep talking past the point. Nobody is claiming we can turn a human mind into a literal bitstring and boot it up like a computer program. That was never the argument. The bitstring analogy exists to make a simpler point: everything that exists and changes according to physical law can, in principle, be represented, modeled, or reproduced by another physical system. The form does not need to be identical to the brain’s atoms any more than a jet engine must flap its wings to fly. The key is not replication of matter but replication of causal structure.

You say we cannot reproduce the brain. But that is not the point. The point is that nothing about the brain violates physics. It runs on chemical and electrical dynamics that obey the same laws as everything else. If those laws can produce intelligence once, then they can do so again in another substrate. That makes the claim of impossibility not scientific, but emotional.

You accuse me of misunderstanding neuroscience and cognitive science. The reality is that neither field understands itself. We have no complete model of consciousness. We cannot explain why synchronized neural oscillations yield awareness. We cannot define where attention comes from or what distinguishes a “thought” from a signal cascade. Cognitive science is still arguing over whether perception is bottom up or top down, whether emotion is distinct from cognition, and whether consciousness even plays a causal role. That is not mastery. That is the sound of a discipline still wandering in the dark.

You act as though neuroscience has defined the boundaries of intelligence, but it has not. We do not have a mechanistic understanding of creativity, emotion, or reasoning. We have patterns and correlations, not principles. Yet you talk as if those unknowns justify declaring machine intelligence impossible. It is the opposite. Our ignorance is precisely why it cannot be ruled out.

Emotion is not magic. It is neurochemical modulation over predictive circuits. Replicate the functional dynamics and you replicate emotion’s role. Creativity is recombination and constraint satisfaction. Replicate those processes and you replicate creativity. Reasoning is predictive modeling over structured representations. Replicate that, and you replicate reasoning. None of these depend on carbon. They depend on organization and feedback.

You keep saying that the brain cannot be “reproduced as bitstrings,” but that is a distraction. Nobody is suggesting uploading neurons into binary. The bitstring argument shows that any finite physical system has a finite description. It proves that cognition, like any process governed by law, has an information theoretic footprint. Once you accept that, the difference between biology and computation becomes one of scale, not kind.

You say LLMs are not creative, not emotional, not reasoning. Yet they already produce outputs that humans classify as empathetic, sarcastic, joyful, poetic, or analytical. People experience their words as creative because they combine old ideas into new, functional, and aesthetic patterns. They reason by chaining relationships, testing implications, and revising conclusions. The fact that you can recognize all of this in their behavior proves they are performing the surface functions of those capacities. Whether it feels like something to be them is irrelevant to the claim that they can reproduce the function.

And now your final claim, that whatever becomes intelligent “will not be an LLM.” You have no basis for that certainty. Nobody knows what an LLM truly is once scaled beyond our comprehension. We do not understand how emergent representations arise or how concepts self organize within their latent spaces. We do not know if some internal dynamic of this architecture already mirrors the structure of cognition. What we do know is that it learns to compress the world into predictive patterns and that it develops abstractions that map cleanly to human meaning. That is already the seed of general intelligence.

You are mistaking ignorance for insight. You think not knowing how something works grants you authority to say what it cannot become. But the only thing history shows is that such confidence always looks ridiculous in retrospect. The physics of intelligence exist. The brain proves it. And the LLM is the first machine that begins to display those same emergent behaviors. Saying it “will not be an LLM” is not a scientific claim. It is wishful thinking spoken from the wrong side of the curve.

[heyjamesknight]

Look, mate, you can keep jumping up and down about this all you want. But you're arguing science fiction at this point. Not really worth continuing the conversation, but thanks.

Best of luck.

[ninetyninenine]

Calling this “science fiction” isn’t just dismissive, it’s ironic. The discussion itself is science fiction by the standards of only a few years ago. Back then, the idea that a machine could hold a coherent philosophical argument, write code, debate consciousness, and reference neuroscience was fantasy. Now it’s routine. You are literally using what was once science fiction to declare that progress on LLMs has ended.

And calling that “science fiction” again isn’t a rebuttal, it’s an insult. You didn’t engage a single argument, you just waved your hand and walked away. That isn’t scientific skepticism, it’s arrogance disguised as authority.

You can disagree, but doing what you did is manipulative. You dodged every point and tried to end the debate by pretending it was beneath you. Everyone reading can see that.

[heyjamesknight]

I’m pretty sure everyone reading can see which of us is the arrogant one.

Good day, sir.