[nicklecompte]

The bigest difference is that training does not change the size or architecture of an artificial neural network, but biological evolution dramatically changes the size and architecture of animals' brains.

Your comparison is sincerely vacuous. It vaguely makes sense if you're talking about GPT-3 to GPT-4 (though I don't think it's helpful). It makes no sense if you're talking about training a single neural network.

[mewpmewp2]

I mean that exposure to a lot of training material yielded in the final set of capabilities. Pigeons and their ancestors were exposed to certain situations throughout evolution that yielded in the formation of neural network and its ability to "count". Which I believe is not actual "one, two, three", but just the amount of signals being activated resulting in a certain output from pigeon. There's a difference in how a human counts, except for small numbers which you can intuitively immediately come up with a number.

There was training material which were situations to which organisms had to produce output for and if the output was good their genetics survived, eventually forming the neural network that was able to handle this training material well, but similarly producing emergent behaviour like being able to "count".

But GPT-Vision can easily do as well what a Pigeon can. What's the exact thing that implies Pigeon is doing it somehow more intelligently?

If you ask them on a picture the quantity of something, I'm pretty sure both respond to the amount of this type of signal received either though light waves or pixels encoded for GPT.

[RaftPeople]

> Aren't animals trained to do all of those things through evolution? Similarly how GPT is trained.

If you interpret this question at the most abstract level of "aren't both solutions arrived at through training/trial+error method?" - then the answer is probably yes, they are both arrived at in some conceptually similar manner.

But they are two very different underlying systems and we don't really understand the biological systems well enough to even be able to truly compare.

Beyond that, it seems that humans (switching to humans from pigeons) have some sort of representation/understanding of the world around us such that even if we produce the same result as ChatGPT to a counting question, the information stored within our systems is not equivalent.

[mewpmewp2]

> aren't both solutions arrived at through training/trial+error method?

But also an underlying neural network type of structure that takes in input, and produces output and changes underneath to then have emerging capabilities (like the counting).

> But they are two very different underlying systems and we don't really understand the biological systems well enough to even be able to truly compare.

Beyond that, it seems that humans (switching to humans from pigeons) have some sort of representation/understanding of the world around us such that even if we produce the same result as ChatGPT to a counting question, the information stored within our systems is not equivalent.

What is the reason to believe that the way pigeons count is anything other than it responding to certain signals

1. Light waves coming as input.

2. Some transformation layers that will abstract the input further.

3. Then pigeons do not really count as people do, but they just respond to the rough feeling of "quantity" or amount of signal received. Because as I understand the studies prove the ability to "count", by them having to just differentiate between counts, and getting rewarded if they are able to do it.

And GPT-Vision can easily do similar things. I can give it an image and ask how many objects are there, and up to an amount it can answer correctly given the image is clear enough.

Similarly pigeons didn't have 100% accuracy in counting. So they are not doing the "one, two, three", they are just seemingly responding to "amount of signal" to me. Similar to how we would be able to tell that certain sound is louder than the other, we are not actually counting the frequencies of the sound. We do not even know what produces the sound. We just decipher that one signal is louder than the other one.

Pigeons after being trained to respond to certain amount of something will associate a strong signal from there with that reward. This seems like what a very basic machine learning algorithm can handle, even more basic or smaller in scale than an LLM. So what makes an animal smarter then?

[RaftPeople]

> But also an underlying neural network type of structure that takes in input, and produces output and changes underneath to then have emerging capabilities (like the counting).

Sure, at an abstract level you could say that, but it requires such a level of abstraction that comparisons don't really mean much. The differences in how the systems function could cause significant differences in underlying functionality and emergent behavior.

For example, some differences when you get into the details:

1-Biological brains have astrocytes that manage the synapses and activity of neurons, constantly and dynamically bringing together different and changing populations of neurons to perform functions (inhibiting some and enhancing activation in others).

2-Neurons aren't the only computational units, astrocytes are also computational units involved in (at minimum based on recent studies) learning and object recognition.

3-Some cells like Purkinje cells learn patterns even when isolated. Something within the cell is learning/storing information about timed patterns of activity and can respond appropriately when the pattern is re-encountered.

4-Dendrites frequently perform preprocessing on signals prior to the signal being forwarded to the neurons soma.

5-Rabbit's olfactory learning and memory is interesting, read up on it if you get a chance. A neuroscientist expert in that field has a theory that matches the data that the network within the olfactory region goes through a minimum-energy type of reconfiguration each time a new scent is detected that is related to some positive or negative. This is interesting from the perspective of how do brains get dynamically reconfigured with learning.

[mewpmewp2]

> 1-Biological brains have astrocytes that manage the synapses and activity of neurons, constantly and dynamically bringing together different and changing populations of neurons to perform functions (inhibiting some and enhancing activation in others).

Wouldn't this be something that could be mirrored or performed even better by just having more layers of neurons in the network. Of course in addition you could have multiple LLMs doing such work together of selecting best optimised systems for a given work. But it seems like astrocytes based on my limited knowledge are more for the reason of maintenance for biological systems which LLMs wouldn't necessarily have to deal with in the first place. So I'm not sure what kind of advantage astrocytes exactly would bring.

> 2-Neurons aren't the only computational units, astrocytes are also computational units involved in (at minimum based on recent studies) learning and object recognition.

But again - what benefits would they provide more over extra layers or LLMs that act together according to an orchestrating LLM?

> 3-Some cells like Purkinje cells learn patterns even when isolated. Something within the cell is learning/storing information about timed patterns of activity and can respond appropriately when the pattern is re-encountered.

But neural networks can in general learn patterns when isolated. It also seems they are more for physical movement, which we should care more about when building robots rather than text based intelligence. Although it seems like for physical movement there's other blockers, like materials. It's all data structures that take input and produce output, which they receive feedback for whether it worked out well and adapt accordingly. I'd assume if LLMs neural networks were given a look, there would be many pockets like the Purkinje cells.

> Dendrites frequently perform preprocessing on signals prior to the signal being forwarded to the neurons soma.

Again it seems like extra layers of neurons. Because I assume with LLMs and other ML tools in addition layers will start to converge on specific set of processing and functionality as they train more. Preprocessing is just a way to make a larger task into more smaller subtasks.

> Rabbit's olfactory learning and memory is interesting, read up on it if you get a chance. A neuroscientist expert in that field has a theory that matches the data that the network within the olfactory region goes through a minimum-energy type of reconfiguration each time a new scent is detected that is related to some positive or negative. This is interesting from the perspective of how do brains get dynamically reconfigured with learning.

I should

But I mean overall, it all seems still the same concept, just orchestrated differently in certain ways and in biological sense it seems it has had to tackle problems that an LLM hasn't really had to, as it has had billions of years to evolve those layers of different systems, but having also a lot of tooling within it to deal with environmental limitations.

So it seems, that given enough computing power, we should be able to make something that is more intelligent than a human, which I do think GPT-4 already is in so many things.

I also am not sure what exactly would GPT-4 have less intelligence in compared to any animal. If you give it the proper input it should be able to perform at least at level of any animal, maybe not with same speed - as animals and as you mentioned in general there are many neural networks within human and animal bodies that correspond only to certain function and are optimised for that specifically.

[RaftPeople]

> Wouldn't this be something that could be mirrored or performed even better by just having more layers of neurons in the network. Of course in addition you could have multiple LLMs doing such work together of selecting best optimised systems for a given work. But it seems like astrocytes based on my limited knowledge are more for the reason of maintenance for biological systems which LLMs wouldn't necessarily have to deal with in the first place. So I'm not sure what kind of advantage astrocytes exactly would bring.

Regarding: astrocytes and cell maintenance vs computation: The picture is getting more complex at a steady pace as scientists learn more. Now they know that astrocytes wrap around synapses (the "tripartite" synapse), detect and emit neurotransmitters and gliotransmitters, have internal calcium signaling and are involved in learning and object recognition.

Regarding: Wouldn't this be...more layers of neurons...": Possibly, maybe probably. Those examples didn't really describe any functional capability, they just described how different our machine is from many people's understanding of how our brain works, which helps illustrate why comparisons to something like ChatGPT are difficult.

> So it seems, that given enough computing power, we should be able to make something that is more intelligent than a human, which I do think GPT-4 already is in so many things.

When you say "enough computing power" do you mean just expanding ChatGPT's number of parameters and training set? I don't personally think that will do the job, I think the key is identifying and providing the specific functional capabilities that our brain utilizes. And I think that requires an approach that is different than just expanding the size of the network and training.

> I also am not sure what exactly would GPT-4 have less intelligence in compared to any animal.

Do you mean ChatGPT's current capabilities? If so, animals model the 4D environment they exist in, ChatGPT is obviously limited in areas like that. Those internal models can be key for some types of knowledge.