Hacker News new | ask | show | jobs
by tegeek 2211 days ago
Comparing Human Brain with a CPU is misconception. In the past when we didn't have digital computers we used to compare Brain with other machines. And now with a CPU. A Brain from a primitive neuron to higher level is not comparable to any machine at all including the CPU.
6 comments

Computers are mathematical concepts, Turing machines being one such concept. Whether computers are implemented using silicon, or oil, or using neurons, it doesn't really matter as we have a mathematical framework for describing abstract machines, and we can determine what is a machine, and what is not.

We did not have this mathematical framework before the age of Turing, Church, Russel, et al.

This doesn't mean that brains are very similar to CPUs, they are not, just like they were not similar to mechanical machines before.

Yet we do now have a way of studying the similarities they have.

Comparing Human Brain with a CPU is misconception. no it is not. Yeah architecturally they are very different and CPU are arguably more programmable / general and less efficient.

What does matter is whether CPUs are theoretically able to achieve all the things that a brain can do (and even more) And indeed CPUs as turing complete, programmable machine are a strict superset of what brains can do. The gap between what task and at which accuracy a brain achieve vs a CPU is decreasing each year as you can contemplate on the paperswithcode.com leaderboards. The difficulty is in software, hardware through clusterisation has arguably order of magnitude more compute than a brain has.

There are four big missing pieces to match human brain performance:

1) Matching its pattern recognition abilities I believe that current statistical learning techniques of SOTA neural networks actually outperform humans on learning continuous data. But humans outperforms by far current software at zero/few shot learning on sparse/discrete data (where gradient descent is not applicable) I believe humans have this performance edge because of 2), 3) and 4):

2) humans can encode and decode meaning with great accuracy in a high level, descriptive complete declarative language called natural languages. They are in many ways far superior to current GQL/datalog/SQL DB languages at encoding and retrieving meaning (that is an isomorphic description of a denoted thing). The field of semantic parsing (+ question answering from the parsed knowledge) is the key to general language understanding and crucially lack funding. Once machines will be able to understand language and retrieve all the knowledge of say Wikipedia, they will be able to transcend human performance on many intelligence/erudition tasks.

3) humans seems to be able to do meaningful runtime code generation.

That is you can develop on demand new solutions to new problems: such as https://www.kaggle.com/c/abstraction-and-reasoning-challenge The field of specification and implementation generation is too underfunded.

4) is the observation that 3) is probably a necessary key for unlocking 2) and that both 2) and 3) are needed to achieve this communication/feedback loop between high level semantic reasoning and statistical operations.

As we can see, humanity overfocus funding on 1) despite being the most solved of all others necessary foundation's to achieve AGI and hence, as a side effect, empirically prove that CPUs superset brains

"And indeed CPUs as turing complete, programmable machine are a strict superset of what brains can do."

This is a fundamental assertion that I do not believe you can make.

The brain cannot simulate a turing machine. It does not have infinite memory, which is a requirement for a turing machine. It can, however, stimulate a linearly bounded automata.

It is also not implicitly obvious that a turing machine can simulate a brain. The primary difficulty that I do not yet see a way around is the fact that a turing machine, which has as its control unit a finite State machine, is bound by the finiteness of those states (finiteness of representation, not of number). The brain has no such constraint. It is analog, and therefore infinite in State representation.

In my opinion, this is more akin to the P versus NP problem, and that we know what needs to be equivalent in order to say that P equals NP, but no one has proved it or disproved it yet. That's how I feel about the statement about Turing machines and the brain. I do not believe we can be dogmatic on that aspect yet either way. We may have opinions, just as we may have opinions about P vs NP, but we must also be careful about stating what is provable and what is opinion, and that is all I'm trying to do.

Of course, I am willing and very interested to gain more insight in this area, so discussion is welcome!

> The brain has no such constraint. It is analog, and therefore infinite in State representation.

This is a common misconception.

I'm sure you are aware that analog signals can be approximated by digital values -- a 10 bit ADC will read a channel to one part in 1024, etc.

You might say that even a 64 bit representation is a poor approximation of a real life signal, which is a real number with infinite precision... But it isn't.

The brain operates at about 300 Kelvin, and so there's a noise floor to all analog signals of about that times Boltzmann's constant, or 10^-20 J. If a neuron impedance is 1 ohm, and at a bandwidth of just 10 kHz, the thermal noise is about 1 nV. For a membrane potential of 100 mV, that's a maximum possible noise to signal ratio of one part in 100 million, which is 26 bits.

Now the brain could depend on the signal below the noise floor, but if so those would be extremely fragile operations, and you could get the same thing on a computer by padding your numbers with random data.

Given how robust a brain is against noise, I'd be surprised if any brain signals are more precise than an equivalent of 3-4 bits.
I agree, and I think in practice the brain's noise floor is also much higher than the theoretical thermal-noise minimum. But I guess the main point is that once we acknowledge that even 32 bits is more than enough, the difference between an analog and digital machine loses a lot of its philosophical weight.
> The brain cannot simulate a turing machine. It does not have infinite memory, which is a requirement for a turing machine.

In practice we call modern computers turing-complete even though they don't have infinite memory. The brain can simulate such a machine.

> The brain has no such constraint. It is analog, and therefore infinite in State representation.

If this mattered, then it would mean analog computers are more powerful than digital computers and therefore the Church-Turing thesis is wrong

Regarding the Church-Turing thesis, it is exactly that, just a thesis. Again, akin to P vs NP. It seems to hold for most cases, but is not proven.

The reason that it's difficult to apply in regards to the brain is that we don't exactly know how the brain is computing... or if it "computes" at all! To my knowledge, we don't have a model of computation for consciousness, emotion, free will, Etc.

Perhaps these are better classified as emergent Behavior rather than computation, but if that is the case I still don't know of a model explaining what computations or rules give rise to the emergent Behavior.

Perhaps the problem is in our definition of computation and what it means to compute.

We do know that the cardinality of the set of possible computational problems is larger than the cardinality of the set of all possible Turing machines. This is provable by simple diagonalization proofs.

The question, then, is whether or not the computations of the brain fall Within the set of Turing recognizable languages (computational problems). To my knowledge, this has not been shown.

As far as I understand, the prevailing opinion is that the brain is a physical object and that its operation does not involve currently-unknown laws of physics (because we have a good understanding of what happens at the scale of an entire atom or above).

A Turing machine can run a simulation based on such physical laws to any desired level of precision (which is enough, because as mentioned in TFA, processes in the brain aren't individually very precise). This is true because of the nature of these laws, which are mostly just asking you to integrate differential equations. If you accept this, then it should follow that a Turing machine can in fact simulate a brain: just run a physics sim on a brain's initial state.

(I do realize that this is far outside the realm of what's doable today, but it seems to provide a solid justification for why it's conceptually possible).

> that its operation does not involve currently-unknown laws of physics (because we have a good understanding of what happens at the scale of an entire atom or above)

Well, we know certain approximations of those laws. Purely theoretically, it is possible that the exact laws at some level of detail that we have not yet been able to observe involve functions that are not computable by a Turing machine, and then it is theoretically possible that the brain itself is computing functions which are not computable by a Turing machine (this would of course assume that the Church-Turing thesis is actually wrong).

As long as the Church-Turing thesis is not proven, we can't say with absolute certainty that the physical world can be simulated to any level of detail by a Turing machine.

Furthermore, even if the Church-Turing thesis was proven, is it possible that the physical world involves transformations that are not even computable at all (even if they can be approximated by computable functions)?

Just to be clear, I do not believe these things. But it is fun to think about the limits of our knowledge.

"any desired level of precision" is actually the issue. The moment you choose a level of precision, you cease being accurate (at that level). If you make the argument that a TM has infinite memory, and can therefore represent an infinite precision, then I would counter that our current defintion of a TM requires a finite tape alphabet (and finite number of states), which is part of the TM's known computational limitations. And, of course, the moment that you use any finite set of symbols to represent an infinitely precise value, you fall into the problem that the set of real numbers has a larger cardinality than the set of possible turing machines (again, simple proof via diagonalization).

It is possible that the brain's imprecision (I would argue that "inconsistency" might be a better word) is a requirement of it's computational ability. Again, we haven't defined how the brain computes, nor do we have a model for explaining its computation, encoding or representation of knowledge, or emergent behavior. We have observed phenomena related to some of these things, but we are far from understanding it. It may be that the computational processes are dependent on the surrounding environment. We know that the biological processes are influenceable by the physical world, but we do not know much about how these external forces affect, limit, or are required for, the process of brain computation.

The quantum world may play a part in consciousness (or no, we don't know). Non-determinism may play a part. It is possible that, in order to simulate a brain, one would have to simulate the entire universe around it in order to predict the behavior... meaning that it may well require a universe to perform the simulation.

Which brings us to a related theory of whether or not we are living in a simulation, but I digress... :)

Isn't the recent Google quantum "supremacy" experiment evidence against the extended Church-Turing thesis?
No, quantum computers as we understand them can be simulated by a turing machine
The extended Church-Turing thesis which I specifically referred to concerns efficient simulation, not just whether it can be simulated.
Google has not proved quantum supremacy, it is a scam. They have proved the truism that running a physical system is faster than running a simulation of a physical system...
https://www.nature.com/articles/s41586-019-1666-5

What part of the experiment in the paper released did you feel like was inadequate?

I mostly agree with your post but:

> The brain has no such constraint. It is analog, and therefore infinite in State

Not necessarily infinite. A lot of people believe that nothing in the world is truly infinite (just very large/small). Infinite quantities in mathematics are just approximations that simplify calculations.

If you go to a sufficiently high precision, neurons and their communication are discrete - the number of neurotransmitter molecules and ions transferred across any synapse is countable, so the number of states (even if we ignore noise and noise tolerance, which we shouldn't ignore) is finite.
The big question is whether a CPU can emulate a brain with the same or better efficiency.
Turing completeness isn’t necessarily an interesting thing to have in common. Many (very simple) models of computation are Turing complete but have vastly different properties. Take for example a cellular automata, a Turing machine, Wang tiles, (cyclic) Tag systems, Fractrans, Register machines, string rewriting systems. All of these are Turing complete. Yet they are miles apart in how they carry out computation. In order to understand and do what the brain is doing we have to figure out the brains model of computation. It will also be Turing complete but it will look very different than a Turing machine.
> CPUs as turing complete, programmable machine are a strict superset of what brains can do

In what way can this be proven?

It's very tempting in an era of tech-centered growth to think of computers as the solution to everything, but we are barely even beginning to understand the brain. We know computers fairly well and can talk about them, but how can we make such a claim when we don't know the other thing we're talking about?

In fact, the brain created the computer, didn't it? Therefore, from that standpoint it is arguable that the brain is a superset of the computer. It's not something I really believe in (because my opinion is that you can't really equate things that are of entirely different units, one of which being unknown), but just a "devil's advocate".

The argument isn't "something like, or a little better, than current CPUs can perform everything a brain can," but something more like "a turing machine can perform everything a brain can or more." This is more an ontological exercise, not an empirical one. If you reduce everything to a "black box" model with inputs and outputs, then sure, the mathematical abstractions of theoretical brains and theoretical CPUs have a congruence. Most objections to this seem to resolve around qualia being something not modelable in machines, but I'm skeptical of that claim.

Can an "arbitrarily advanced computer do everything a brain can do?" Empirically, right now, current machines can't but we are talking about "future machines, via line-of-sight extrapolation". Not fundamental leaps in tech, but incremental ones. It seems plausible, but it seems we expand the depths of the complexity of the requirements nearly as fast as we advance current capabilities. I don't know, but I'd put my money on the technology catch up.

Being skeptical of the claim that a certain qualia is not modelable in machine is just as valid as being skeptical of the exact opposite. This is exactly why I asked if there was anything beyond what the original poster said. Without it, a post based on the exact opposite assumption could have been written and considered just as valid.
Fair criticism, I didn't tackle that head-on. The following doesn't actually make a cogent argument either, but I'll elaborate that my intuition is that qualia (conceived as something nearly tangible) are more like "the soul" or "spirits" and that, as such, thinking they exist in the brain or a turing-machine is nonsense. To the extent they are more like some combination of memory and emotional-stimuli, then they just represent a particularly interesting set of internal states, but are still something that can be mathematically modeled.
> In what way can this be proven?

Proven? Nothing in science is ever proven.

But on half a millenium we have failed to find anything that can't be simulated by math, and Turing completeness means a computer can simulate anything that can be simulated by math. We also can simulate all the smallest components of a brain.

At this point the claim that math can not simulate it is highly extraordinary.

> Turing completeness means a computer can simulate anything that can be simulated by math

Technically, it is not proven that Turing machines can compute all computable functions, so there is some purely theoretical possibility that the brain could be able to compute functions that a Turing machine can't.

Personally I find that extremely unlikely, and agree that it would be extremely surprising. But it wouldn't invalidate anything we have proven so far.

It would imply that our brains are using currently-unknown physics, since all current theories are computable.
We have not been able to simulate any aspect of subjective, conscious experience using a mathematical model, and personally I think we have no good reason to believe we ever will. The qualitative, by definition, cannot be quantified.
I am not convinced of the usefulness of this comparison.

The first of your big missing pieces starts from the best that we have been able to achieve with computers so far, and while its completion might be a big step in computing, it would not necessarily be a big step in understanding the human brain - after all, quite primitive animals have impressive abilities in this regard. Using the best computing has done as the yardstick for quantifying the human brain's ability is the wrong way round.

The remaining missing pieces are vague, with no clear indication that they fit into the brain-as-CPU model. For example, while it is true that "[human languages] are in many ways far superior to current GQL/datalog/SQL DB languages at encoding and retrieving meaning (that is an isomorphic description of a denoted thing)", this vastly understates the capabilities of language. Once again, you are using current technology as the yardstick, with no basis for assuming that it is of the right scale.

Overall, you seem to be assuming that the rest of the puzzle is almost within reach. That is certainly a logical possibility, but not one with a great deal of objective evidence in support. FWIW, my opinion on the matter is that we probably don't even know, in any well-defined way, all the questions to be answered.

Even if we grant the premise that a suitably-programmed computer (not just a CPU) could have capabilities that are a superset of those of a human brain, that would not necessarily justify saying one is very like the other - that would be like saying a dynamo is a solar cell because they both produce electric current.

I agree. For some reason 2) and 3) reminded me of the book "The mating mind" https://en.wikipedia.org/wiki/Geoffrey_Miller_(psychologist)...
“programmable machine are a strict superset of what brains can do”

As others already replied, that’s a statement that isn’t universally accepted to be true.

As an example, there’s consciousness. People disagree about whether it exists, whether it’s (fully) ‘in’ the brain, and on whether computers could in theory be conscious.

There are people who answer those questions with yes, yes, and no, and, since we don’t even have a good idea about what consciousness is, one cannot reliably argue that they are wrong (also not that they are right, of course)

Before CPUs existed, we would compare brains to steam engines. There was a very interesting article posted here on HN a while ago, explaining why humans always pattern match their understanding of the "mind" (or "soul") to whatever technology is fashionable in their time: steam engines, computers, etc. It also explained the pitfalls of doing so.

I think there is at this time no indication human brains are in any way similar to CPUs. It might be interesting to consider the question, of course.

But steam engines and hydraulics and gear mechanisms are all Turing complete. There is nothing wrong with those models. You could build a brain out of any of them, unless the brain computes something that is not computable.

If the brain does something that is not computable, that's a direct challenge to some of our most established science. It is possible, but I think very unlikely.

> You could build a brain out of any of them, unless the brain computes something that is not computable.

Could you? That's sort of begging the question. We do not know if something "Turing complete" can be used to build a brain like the human brain. That's precisely the point.

> If the brain does something that is not computable, that's a direct challenge to some of our most established science.

A challenge for computational neuroscience maybe. Otherwise I don't see the challenge for neither neuroscience nor computer science. If someone wants to make the claim you can build a human brain out of something Turin-machine-like, that's an extraordinary claim, not established science.

The argument I'm responding to is one that says people are wrong about brains being computers because people always believe they can make brains out of technology of the day. My point is that all of these things are the same theory, and it is one that has not been disproven.

If a brain cannot be produced in a turing machine, it must perform some non-computable activity. That would mean physics cannot be accurately simulated in a computer, which I believe would be earth-shaking in that world. That brains can be reproduced in a simulation is a default assumption, that something composed of molecules can produce outcomes that cannot be computed is an extraordinary claim, for which, I believe, there is no evidence.

To be fair, CPUs are Turing machines. That makes them much more comparable to anything that mainly does information processing than to anything else.
I think the danger is that it's always "obvious" that the current fashionable tech works in analogous ways to the mind/brain. We can spend all day finding ways in which they are similar; for example how the brain does information processing and the CPU does too.

The point is, I think, people from the steam engine era had similar reasons why the mind/soul was exactly like a steam engine. I won't try to reproduce them here, but I'm sure there were convincing arguments at the time. Who has the awareness to claim, before the current fashionable technology becomes unfashionable, that maybe no, the brain is not a close match for an information processing machine? ;)

I thought it was about similarity of simulated neurons, not the CPU itself.
> What does matter is whether CPUs are theoretically able to achieve all the things that a brain can do (and even more) And indeed CPUs as turing complete, programmable machine are a strict superset of what brains can do.

It is not proven in any way. Turing's postulate is just a postulate, it is not even a theorem, just a conjecture. And AFAIK it cannot be proven, actually.

Is there anything analogous to software in biology?
Biology is the ultimate legacy software running on one of the oldest platforms ever developed, the organic compounds. It is literally a giant genetic algorithm to write instructions (DNA) for manufacturing molecular machines (proteins) that interact with each other in an extremely complex graph of relations (protein pathways, i.e. control flow).
That feels more to me like hardware and software in the sense of a Jacquard loom. I suppose it fits though.

I was thinking more about what's going on in the brain. We have all the regions mapped to specific functions with higher and lower level parts. The low level parts seem to be like hard-wired stimulus-response mechanisms. Are the higher level systems the same at a meta level or is there a type of program running on the hardware of the brain?

The stimulus response mechanisms are far from hard wired. The brain is plastic at all levels.
A baby doesn't need to be taught how to breath or cry. That seems pretty hard wired.

Anyway, that wasn't really what I was asking about. Is there any separation between the biological hardware of the brain and the instructions of software?

This a very simplistic view, based on assumption that the world is discrete. The whole idea of software relies on the concept of digital computer, a discrete machine. The world might indeed be analogous and real numbers might actually exist.
If world did run on real numbers that we could harness for computation I would be more than happy, because using those we would be able to perform hypercomputation. See https://en.m.wikipedia.org/wiki/Real_computation

However this is forbidden by Bekensteins bound, so unless modern physics is horribly broken it’s ruled out at least in any sense visible to us even in principle.

Not a quantum physicists, but IMO Bekenstein bound is not applicable here, because quantum laws are non-deterministic, therefore you can describe the structure of a system, but you cannot describe how it will evolve. Quantum randomness might be in the very essence of how the brain and mind works.
DNA and proteins are obviously discrete, so even if the hardware relies on some fundamental analogous behavior, the 'software' and each hardware component can still be analyzed as discrete behavior.

However, for anything operating at human temperatures we can reasonably assume that any effective behavior can be simulated by discrete operations, as any nuances of fundamental analogousness would be drowned by thermal noise and the amount of precision that any behavior can require is rather low, much lower than e.g. any standard floating point number in a discrete CPU.

I think the assumption that software may only be digital is the limited one.
Otherwise it becomes a meaningless, all-encompassing term.
I am sure that I saw this exact message on HN before. Did you copy it from someone else or did you repost your own post?
I wrote it out ad hoc, I don’t doubt that something similar has been written before though.
The human brain isn't Turing-complete.

Turing completeness implies infinite recursion, which the brain obviously can't do.

This is a silly objection, trivially because obviously no finite physical system - brain or computer or whatever - can be constructed with the storage equivalent of an infinitely long tape. But if you allow for the fact that humans can do things like write things down and share information with other humans and build computers to store information, our information processing capacity is not limited to the set of states we can hold inside the atoms inside our head.

But also, the claim lacks evidence: We’ve never seen a human being yet whose program didn’t eventually halt.

That doesn’t mean the hardware isn’t capable of running a program that never halts, just that we haven’t found such a program yet.

Indeed if you consider human mindware as a whole, given that when humans reproduce they create new copies of the mind running in new bits of hardware... maybe Human minds are infinitely recursive after all?

Technically Turing completeness requires infinite memory for that (or an infinite tape if we're talking about the original turing machine concept), which no Turing-complete machine has. In other words, the brain is as Turing-complete as any machine that we also consider to be so. We'll always be bounded by limited memory and limited time.
We do not know if human brain is indeed Turing-complete, or even if it is a Turing machine at all. Human Mind certainly is, but if brain is or not we do not know.
While I agree that comparing a human brain or mind to a Turing machine is not helpful, the objection you make here is less significant than it first appears.

There is a subtle difference between unbounded recursion, which a Turing machine is taken to be capable of, and the actual ability to achieve infinite recursion. In no application of a Turing machine, either as an actual physical device or as a hypothetical one in a logical argument, is it ever required to perform infinite recursion, which would just be one way of not halting.

For all practical and theoretical purposes, what matters is that the machine being considered does not exhaust its ability to recurse while performing the computations being considered. Consequently, the standard practice, of saying that computers and certain other devices are Turing-equivalent, with the usually-implicit caveat of being so up to the limit of their recursive ability, is both reasonable and useful.

> For all practical and theoretical purposes, what matters is that the machine being considered does not exhaust its ability to recurse while performing the computations being considered.

You're right, and thanks for the more strict definition.

Regardless, the 'recursion limit' of the human brain is really low. (Say, seven things at once or thereabout; not going to links proofs but it's a non-controversial statement.)

Certainly not enough to implement any sort of computing machine. Human brains are notoriously bad at arithmetic and state machines.

Why is that obvious? My brain’s been infinitely recursing for years as far as I know
That's what the article does though. And there are experiments trying to simulate parts of brains but we realize that it's extremely hard to do that and we are very far away from simulating even a mouse brain.
The difference is that CPUs, unlike those other machines, can be used to model/simulate things that are similar to brains. There is impedance in the translation, of course, but that impedance can be measured as a sort of “distance” between the architectures; just like one might measure the “distance” between two Instruction Set Architectures.
"...the question of whether Machines Can Think, a question of which we now know that it is about as relevant as the question of whether Submarines Can Swim."

Edsger Dijkstra, EWD898, 1984

Whether or not it's comparable depends on the level of distinction you're trying to make. Obviously, CPUs don't think or experience the world (but on the other hand that kind of "feature" seems increasingly likely to be implementable in software, even if our current CPU architectures are rather unsuitable for that goal). However, if we're gonna talk about energy efficiency and computation performance, now that it has become evident that the brain is merely a kind of a computer, we can definitely look for parallels.
> now that it has become evident that the brain is merely a kind of a computer

I am ignorant in this area. But I keep reading how brains are nothing like computers the more we learn. Your statement seems to suggest otherwise and id love to read about it. Can you drop something where I can start exploring about how the brain has become more evident that it's merely a kind of computer? Thanks!

The brain is thought to be merely a computer in the original sense of a long strip of paper along with a scribe and a rulebook. The logic is, a Turing machine can simulate quantum electrodynamics to an arbitrary degree of accuracy. Then, two beliefs about physics and the structure of the brain are included:

1. There is nothing going on in the brain that would require simulation to infinite accuracy. Not even a chaotic system would have this property, because they take a finite time to "blow up" an initial uncertainty, and the smaller the initial uncertainty the longer they take to blow up. For this proposition to be violated there would have to be an undiscovered fininite-time nondeterministic blowup, which is unlikely, but I've heard rumblings that we haven't proven that it can't happen in Navier-Stokes. So maybe it can happen in the brain.

2. There is nothing going on in the brain that depends on nuclear physics or anything more "powerful" than quantum electrodynamics.

I have not seen any evidence that 1 or 2 aren't true for the brain, so that puts something behind saying it's "merely a computer."

If you are looking for a book for an introduction, I would suggest Mindware by Andy Clark is pretty reasonable. Pub 2014; ISBN: 9780199828159