[inimino]

> Aren't animals trained to do all of those things through evolution? Similarly how GPT is trained.

No. Animals evolved and are able to do those things. Evolution is not training, and evolution has approximately zero to do with how transformers work.

[mewpmewp2]

What's the difference here?

Both seem to have adaptive neural networks where those networks change as time goes on due to a reward - for animals, mutated genes being more likely to be given forward if the change was good. Over millions of generations it's statistically likely that more good genes that caused the neural networks to be in a state that is better able to solve problems within the environment get passed on, eventually resulting in an emerging intelligence. For training you similarly change the state of the neural network depending on whether the answer is good or bad.

[inimino]

Evolution operates on genes, which do not encode synaptic connections for one thing. The analogy you're making here is so stretched it's hard to begin to say what's wrong with it. Backpropagation and natural selection are about as different as two things can be. About the only thing you can say they have in common is that both can be modeled as optimization processes.

What's the difference between a star and a bonfire? Both use fuel and produce heat and light.

[mewpmewp2]

I mean the point was about LLMs not truly being problem solvers because they were trained to do so as opposed to having been evolved through evolution. I'm looking for what the difference is specifically within that dimension. Biological pigeons had their own process of evolution how they reached to have the type of neural networks and systems in themselves that gave them the ability to count - but not in all contexts for sure.

So yes, my point is that both have an optimisation process that through time lend them those emerging capabilities.

[inimino]

No, the point was that LLMs are not good at problem solving because they are not good at problem solving, not because they were not evolved. We don't understand how we or animals solve problems, which is why we haven't yet succeeded in replicating that in AI. You're the one bringing in these incredibly strained analogies, because you want GPT 4 to be more than it is or something, I'm not sure.

[nicklecompte]

The bigest difference is that training does not change the size or architecture of an artificial neural network, but biological evolution dramatically changes the size and architecture of animals' brains.

Your comparison is sincerely vacuous. It vaguely makes sense if you're talking about GPT-3 to GPT-4 (though I don't think it's helpful). It makes no sense if you're talking about training a single neural network.

[mewpmewp2]

I mean that exposure to a lot of training material yielded in the final set of capabilities. Pigeons and their ancestors were exposed to certain situations throughout evolution that yielded in the formation of neural network and its ability to "count". Which I believe is not actual "one, two, three", but just the amount of signals being activated resulting in a certain output from pigeon. There's a difference in how a human counts, except for small numbers which you can intuitively immediately come up with a number.

There was training material which were situations to which organisms had to produce output for and if the output was good their genetics survived, eventually forming the neural network that was able to handle this training material well, but similarly producing emergent behaviour like being able to "count".

But GPT-Vision can easily do as well what a Pigeon can. What's the exact thing that implies Pigeon is doing it somehow more intelligently?

If you ask them on a picture the quantity of something, I'm pretty sure both respond to the amount of this type of signal received either though light waves or pixels encoded for GPT.

[RaftPeople]

> Aren't animals trained to do all of those things through evolution? Similarly how GPT is trained.

If you interpret this question at the most abstract level of "aren't both solutions arrived at through training/trial+error method?" - then the answer is probably yes, they are both arrived at in some conceptually similar manner.

But they are two very different underlying systems and we don't really understand the biological systems well enough to even be able to truly compare.

Beyond that, it seems that humans (switching to humans from pigeons) have some sort of representation/understanding of the world around us such that even if we produce the same result as ChatGPT to a counting question, the information stored within our systems is not equivalent.

[mewpmewp2]

> aren't both solutions arrived at through training/trial+error method?

But also an underlying neural network type of structure that takes in input, and produces output and changes underneath to then have emerging capabilities (like the counting).

> But they are two very different underlying systems and we don't really understand the biological systems well enough to even be able to truly compare.

Beyond that, it seems that humans (switching to humans from pigeons) have some sort of representation/understanding of the world around us such that even if we produce the same result as ChatGPT to a counting question, the information stored within our systems is not equivalent.

What is the reason to believe that the way pigeons count is anything other than it responding to certain signals

1. Light waves coming as input.

2. Some transformation layers that will abstract the input further.

3. Then pigeons do not really count as people do, but they just respond to the rough feeling of "quantity" or amount of signal received. Because as I understand the studies prove the ability to "count", by them having to just differentiate between counts, and getting rewarded if they are able to do it.

And GPT-Vision can easily do similar things. I can give it an image and ask how many objects are there, and up to an amount it can answer correctly given the image is clear enough.

Similarly pigeons didn't have 100% accuracy in counting. So they are not doing the "one, two, three", they are just seemingly responding to "amount of signal" to me. Similar to how we would be able to tell that certain sound is louder than the other, we are not actually counting the frequencies of the sound. We do not even know what produces the sound. We just decipher that one signal is louder than the other one.

Pigeons after being trained to respond to certain amount of something will associate a strong signal from there with that reward. This seems like what a very basic machine learning algorithm can handle, even more basic or smaller in scale than an LLM. So what makes an animal smarter then?

[RaftPeople]

> But also an underlying neural network type of structure that takes in input, and produces output and changes underneath to then have emerging capabilities (like the counting).

Sure, at an abstract level you could say that, but it requires such a level of abstraction that comparisons don't really mean much. The differences in how the systems function could cause significant differences in underlying functionality and emergent behavior.

For example, some differences when you get into the details:

1-Biological brains have astrocytes that manage the synapses and activity of neurons, constantly and dynamically bringing together different and changing populations of neurons to perform functions (inhibiting some and enhancing activation in others).

2-Neurons aren't the only computational units, astrocytes are also computational units involved in (at minimum based on recent studies) learning and object recognition.

3-Some cells like Purkinje cells learn patterns even when isolated. Something within the cell is learning/storing information about timed patterns of activity and can respond appropriately when the pattern is re-encountered.

4-Dendrites frequently perform preprocessing on signals prior to the signal being forwarded to the neurons soma.

5-Rabbit's olfactory learning and memory is interesting, read up on it if you get a chance. A neuroscientist expert in that field has a theory that matches the data that the network within the olfactory region goes through a minimum-energy type of reconfiguration each time a new scent is detected that is related to some positive or negative. This is interesting from the perspective of how do brains get dynamically reconfigured with learning.