[seydor]

This is a philosophical and epistemological matter that is often undiscussed. Cognitive philosophers are still hung up on questions from the 70s and their offshoots ("hard problem") etc.

On the other hand i am not sure if the "computational" people often know what they are doing. Looking at something like the deep Transformer models, one has to ask if there is any rhyme or reason there, or the thing just works because it's too big and deep. Same even with gradient descent methods, are we sure there aren't closed form solutions instead?

There s an even more pessimistic view of this: that the brain and its creations (language, formal systems etc) are resting on the chaos of spiking cells, and are not as ideal as I 'd like them to be.

[visarga]

Apparently GPT-3 trained on code gained the ability for reasoning, but plain language, or even multi-task fine tuning, were not enough. Maybe the quality of the data makes a difference in the robustness of the model.

This interesting discovery, when applied to brains, means the same brain trained with different data would be able to display emerging abilities. Maybe these abilities are more in the data than the architecture.

If we want better AI we need to come up with better data.

[seydor]

Maybe, or most of human writing is rambling without much effective structure, unlike code