[ainch]

Thank you for expanding! I come from more of an ML background so still learning a lot on these topics:

Agreed that the neocortex uses fewer layers because of looping - I also suspect it's partly because neurons are more complex than the neurons used in ANNs, so in principle they should be capable of more sophisticated computation in a single forward pass (especially considering that handling multiple neurotransmitters could mean superposed functions).

The point about there being one correct way to model something does seem backed up by the Platonic Representation hypothesis (https://arxiv.org/pdf/2405.07987). I've even seem some work that shows you can find a bijective map between the latent spaces of different transformers (https://arxiv.org/pdf/2505.12540).

Your point about Finsler spaces is fascinating - I hadn't come across the term before. It'd be interesting to see if there's work that specifically allows latents to exhibit that kind of directional metric behaviour, and whether that improves generalisation or something.

On the entire brain being dedicated to prediction, that's more of a high-level comment. I was thinking of work like Andy Clark on predictive processing, which suggests that even regions of the brain which we think of as receptive (eg. the visual cortex) may be implementing a generative/predictive model which is corrected by sensory input.

[koverstreet]

I'm not sure if the complexity is at the neuron level. It's clearly possible; we see some pretty complex behavior from single celled organisms - Stentors - and there were the experiments (starting in the 70s, I believe) that conditioned flatworms, ground them up and fed them to new flatworms and showed that the new flatworms had the same conditioning. Those experiments were replicated but never explained, but they hint at something like maybe memories being encoded in RNA.

It'd be wild if our brains used a mechanism like that somewhere, but I doubt it's in the neocortex because those would be slow processes compared to what the neocortex has to do.

I think the sophistication in the neocortex just comes from it using something more sophisticated with transformers. Most of the looped LLM research I've scanned through, they seemed to be training the model to "know" when it should loop more and think harder, and I don't think that approach will ever work - you're solving for paths on a manifold, so the looping needs to happen at the abstraction layer below the manifold, looping more if it's still converging. And architectures with more recurrence are just harder for us humans to reason about and comparatively easier for dumb evolution to find clever solutions, so my money is that that's where the neocortex has LLMs beat.

And I'd say the manifold hypothesis is quite a bit more than a hypothesis at this point. There's been work on production LLMs to use manifold analysis to optimize them - there was a paper in Nature on it back in January. Baby stuff compared to what could be done, but the geometric approach keeps popping up in mechanistic interpretability and seems to be showing the most insights.