Not intended to be snarky, but what would you consider them? Is it akin to a function in the mathematical sense, that takes (sensory) input and creates output based on that? If so, how does this function work, if not by statistics? I am genuinely interested in your point of view. Also: Don't you think humans can be somewhat compared to a "pretrained model", as in human genetics gives the brain a head start, so that it can start speaking latin from what you deam "homo sapiens mumbling?
Not a specialist, but I think each individual is just a small step of gradient descent for the large neural network of humankind.
At our individual scale, we look like a rigid ball of statistics, but at the global scale, we carry a small amount of gradient/delta that pushes humankind in a broader direction.
LLMs have been able to reproduce the former, it is unclear how they can contribute to/replicate the latter.