[ACCount37]

Nah. Physical limits of computation are far enough away that the "constant multiplier of advantage" would have to be measured in OOMs. "Computers can be, at most, 1e11 times more powerful than brains" is not the saving grace you want it to be.

And, so far, the results of "visual data for improving general intelligence" runs were nothing but disappointments.

I think vision is just a piss poor modality to learn intelligence from? Very low value, per bit and per token both. You only ever want to tap it if you need your AI to operate based on visual data at deployment time. Otherwise, even "experience" is best gathered in text RLVR rollouts.

The secret of human sample efficiency isn't that visual data is somehow better for learning intelligence. It just isn't. Human "training data" is a hundred kinds of awful - humans are just good at scavenging it for all its worth. Evolution has tuned that very well.

Which means: AIs can get good at it too. It's not a wall - it's a skill issue.

[blamestross]

I think you are taking an entire space of "intellectual immune system" out of consideration. More than that, you are ignoring the core reason I think they are bottle-necked. They might have more access to compute. But to compute what? The bottleneck of intelligent behavior isn't compute it is experience. We have a lot of "text encoded experience" to feed it, through our collective corpus of writing, but ultimately potential behaviors can only be tested by active experimentation. No amount of observation can discern correlation from causality. Only active experimentation. The "train on itself" only works in a "toy universe" where the model of consequences are trivial to "test"

So in order to scale, AI doesn't need compute. It needs "engagement with reality and agency". Which is STILL might do better than us, but is happening in the real world, with real competition over resources. As long as we don't do something dumb like enthusiastically give it control over our major economic actors. I don't think we need to worry.

On the "intellectual immune system" side, I would argue that language's limitations are themselves fitness. We are already in danger of memetic hijacking. All those points you make about multiple instances of an AI cooperating, don't take malice and memetic attacks into account. It goes back to "why I am not afraid of grey goo". I trust yeast to find a way to metabolize basically everything. We have memetic attacks too.