[layer8]

I don’t think the amount of data is essential here. The human genome is only around 750 MB, much less than current LLMs, and likely only a small fraction of it determines human intelligence. On the other hand, current LLMs contain immense amounts of factual knowledge that a human newborn carries zero information about.

Intelligence likely doesn’t require that much data, and it may be more a question of evolutionary chance. After all, human intelligence is largely (if not exclusively) the result of natural selection from random mutations, with a generation count that’s likely smaller than the number of training iterations of LLMs. We haven’t found a way yet to artificially develop a digital equivalent effectively, and the way we are training neural networks might actually be a dead end here.

[ACCount37]

That just says "low Kolmogorov complexity". All the priors humans ship with can be represented as a relatively compact algorithm.

Which gives us no information on computational complexity of running that algorithm, or on what it does exactly. Only that it's small.

LLMs don't get that algorithm, so they have to discover certain things the hard way.