[ACCount37]

And we can barely figure out how the modern LLMs work.

That doesn't bode well for minds being human-interpretable, not at all.

I used to think that the biggest bottleneck to understanding the workings of the human brain was that it defies instrumentation. Which could be solved by better imaging techniques, high throughput direct neural interfaces, etc. But looking at the state of AI now?

If we had full read/write access to the state of every single neuron in the brain, what would we be able to learn? Maybe not that much.