[cma]

You're talking about something orthogonal, how efficient it is. He's talking about something different:

https://en.m.wikipedia.org/wiki/Catastrophic_interference

Which practically requires full retraining at every step to integrate new knowledge. I think we have some partial solutions like learning to select between finetunings, but not if the task needs to crosscut between them.

The human brain doesn't seem to suffer with catastrophic interference to nearly the same degree, independent of its computational efficiency, though there are possibly related things like developmental stages that if they are delayed may never be able to take place.