[20after4]

This right here. I actually am strongly starting to believe that this is indeed what's going on.

I read the book Kingdom of Speech a few years ago and that also left me with the perspective that perhaps language has a lot more to do with how we think and perceive the world than most people like to admit. The book has been heavily criticized but I believe it made an interesting point about language.

[moffkalast]

I mean hasn't it been proven on multiple occasions that humans who never learn a language during their life also don't develop higher cognitive skills and stay on the level of a child? Clearly language plays a notable part in reasoning, and it's frankly no surprise that we're seeing this behaviour from language models.

It's also quite interesting how foundation models and fine tuning appear analogous to being born with a brain that already has six million years' worth of weights in it (trained implicitly through random changes and natural selection), which are then adapted over the course of a lifetime when environment relevant data is gradually obtained.