Hacker News new | ask | show | jobs
by flak48 3322 days ago
~100 billions neurons in a human brain I guess. Not sure what 1.6KB per neuron is too close to
2 comments

I'd guess that it would be close to the size of the adjacency matrix for the human brain, if it is given that the brain has ~1000 dendrites per neuron.
This comment currently has a reply ( http://i.imgur.com/N7BZJfu.png ) which shows why I wanted to leave my original post as a "riddle" rather than coming out and saying it.

The following "crap" is not relevant to anything, only read it if you want the answer to why 1.6 KB is insufficient for a certain science fiction fantasy which will never, not in ten, a hundred, or a thousand years, become relevant any more than magicians will start flying around on brooms. It has nothing to do with anything. Don't get me wrong and think that I think it has relevance to something.

Original version of this comment:

--

Well, it's clear that you could not come even close to perfectly describing the state of a cell and all its connections in 1.6 KB. As just one example, every cell has a different genome[1] which if you encoded it in full would be 700 MB right off the bat. I'm not saying the fact that every neuron has a different genome is relevant to its computational functions, but 1.6 KB is cutting it very close. Let's explore why this is so.

Suppose we simplified and said that each neuron may be connected to n other neurons (we will calculate what n fits in our memory). Addressing 100 billion takes 36 bits. (You need approx. 36 bits to name a number between 1 and 100 billion, i.e. if all you're doing is naming which other neuron it's connected to.). So let's see how many "synapses" the 1.6 KB might be enough to address. -- we are looking at how many connections could be encoded into 1.6 KB.

So 1.6 KB if you did nothing else but as a simplification say that you were referring to another neuron, then 1.6 KB = 1.6 * 1024 * 8 = 13107 bits, divide that by 36 and you get 364 addresses. Suppose you were to then encode each of those addresses with a single byte (an 8-bit brain) representing the strength of that connection, and you are down to only encoding 13107/44 = 297 neural connections with an 8-bit connection value. [EDIT!! As pointed out in a reply, original version of this paragraph contained a math error - I originally divided by 8 after dividing by 36 -- as though each bit of the address needed a connection strength, rather than adding 8 to 36 to get the number of bits needed for each connection and only afterward dividing that into the 1.6 KB]

Okay, so let's see how "297 neural connections" stacks up.

In fact "Each neuron may be connected to up to 10,000 other neurons", but "Each of the neurons has on average 7,000 synaptic connections to other neurons." This means we do not have 8 bits - we do not have even a single bit. (Because 7,000 is more than 297 or even 364).

So you cannot even store the full address of each synaptic connection (the full address of the other neuron it's connected to) let alone a value of the strength of it, or anything else computationally interesting that may be going on chemically. Of course, obviously one neuron will not be directly connected with another far away, for example on the other hemisphere of the brain (meaning you can almost certainly shave nearly 1 bit of addressing off of the 36 bits from the get-go, and likely you need a far smaller address space, if you really look at it.)

But as you can see, however you slice it, you're just cutting it extremely slow. If synaptic connections had even 10 bits of value of some kind than you're way over your budget here.

But as you can see we're close - the numbers just barely don't work.

HP are on the right track - they simply need to step their game up.

[1] recently in the news - https://www.scientificamerican.com/article/scientists-surpri...

EDIT* - thanks for the correction, yes, there was a math error.

You don't need a single memory machine to simulate the human brain though. Because neurons have a clear spatial structure, you can break the problem up into a mesh, give each piece of the mesh to a different machine, and give them some interconnect so they can broadcast relevant portions of their state to neighboring machines. You don't need low latency direct memory access for every node to all of the memory. Look at something like the https://en.wikipedia.org/wiki/Tianhe-2 which has about a petabyte of RAM

EDIT: I think you're also not considering the ample opportunities to compress this data set. For instance, you can represent the connections between neurons with a sparse adjacency matrix, and there is a whole body of literature devoted to coming up with small representations of that kind of data, while still being able to perform operations efficiently. For the genome variance, keep in mind the variance is going to be absolutely tiny in comparison to the whole genome, so you can just store the genome of one of the neurons and then the diffs between it and all the other neurons or something.

> why I wanted to leave my original post as a "riddle" rather than coming out and saying it.

Because the comment was fundamentally irrelevant? You could post "step up your game" on every single tech submission. It's white noise.

You made a math error:

You can encode about 300 addresses, at 36+8 = 44 bits per item, not 45 addresses, at 36*8 = 288 bits per address.

The bits are additive, because we're storing an address and data, not a byte of data per bit of address.

thanks, corrected.