[gjm11]

A convolutional neural network is really somewhat like a visual cortex. Obviously AlphaZero doesn't literally have a visual cortex -- actual literal visual cortices are features of actual literal brains made out of meat -- but it definitely has something that does something akin to visual processing, in a way that LLMs don't. Or at least they don't on the face of it; maybe well trained large enough LLMs have effectively implemented something kinda-visual-cortex-like on top of the transformer architecture.

(I bet there are people at all the big AI labs working on ways to incorporate something more CNN-like into LLMs somehow.)

[randsorex]

I believe the architecture for convolution neural networks were directly inspired by how vision works and some of the core design choices map onto real features of the visual cortex.

[vatsachak]

That's fair. Two things that are heavily underrated are architecture and encoders