[qz_kb]

"removing biological limits" is a big handwave. Biological hardware is tuned exceptionally well for intelligence, and we have a massive amount of it distributed worldwide. We already remove biological limits by building tools, computers, powerplants etc. Duplicating one AGI unit now doubles the amount of energy and resources it needs, and also creates a requirement that the system can somehow produce more of itself and all the supporting infrastructure it requires. It's not just a matter of being "intelligent" (im not even sure what "intelligent" means in this context either) - it will need to have power to act in the world, and not get sidetracked over-weighting any one of the multiple competing optimizations in complicated real world problem landscapes it will need to navigate to scale.

[AgentME]

It takes decades of training to make new humans useful enough to participate in many subjects of research. Having AGI researchers means being able to duplicate any expert researcher at any time. Getting more computing equipment to run more AGIs costs money but so does training a human for decades. Even if the AGI is more expensive for some time, there will surely be research subjects that today have money to spare and not enough expert humans that will be able to explode in expert manpower through AGI.

>it will need to have power to act in the world

- Human researchers would bring AGI researchers into their teams and empower them to get stuff done.

- AGIs would be competent at making money through intellectual/remote work and have money to use to act in the world and get what they want.

>and not get sidetracked over-weighting any one of the multiple competing optimizations in complicated real world problem landscapes it will need to navigate to scale.

Humans face this obstacle too.