[xyience]

I don't think there's much reason to be confident that the first AGI will be like a low-IQ human, at least for very long, even if it starts off as an emulated human brain. Machines have the huge advantage of faster materials (neurons are slow), perfect memory, perfect calculation, ability to scale horizontally, backups to restore in the event of bad self-modification experiments, and no need for things like sleep and food to take up its time from learning, improving itself, and acting upon the world.

[argonaut]

You've omitted the huge, overwhelmingly outweighing disadvantages of intelligence in machines, which is that they don't work. Given that research progress is incremental, there really isn't reason to believe we will jump from narrow AI to super-smart AGI, instead of narrow AI to dumb AGI to smart AGI to super-smart AGI.

[xyience]

There are historical examples of discontinuities, though I don't think the FOOM debate will be settled soon. It may be quite a while before we even get to "dumb AGI", but the hardest part of that is the G part. Right now "they don't work" is indistinguishable from "they don't exist", but if we get that G, I don't see how you could claim either. From there, even if we suppose it's another huge leap to get to true super-intelligence instead of a FOOM, the time to get to merely smart AGI, and indeed smarter-than-human AGI, would be short, if only for the basic advantages of a silicon machine substrate. If all we had were human brains running on silicon even, that would be enough to quickly reach superhuman general intelligence, even if not true super- (or perhaps ultra- as I.G. Good originally put it) intelligence that we expect for a Singularity event.

[argonaut]

This makes no sense. This is just unsubstantiated speculation right now. There is no reason to believe that dumb AGI will hardware-scale to smart AGI for free. In fact most machine learning algorithms have diminishing (logarithmic) returns with data and compute.