[dwighttk]

>It can also be pointed inward to improve its own effectiveness.

Assumption. Intelligence (which isn’t defined) may be something that can grow without bound, or it may be something that plateaus just above the brightest human yet (again this is ill defined. “IQ is a number. There are numbers that are higher, so intelligence must be able to grow” is about as much thought as some people put into it) or maybe it is something that can grow without bound, but the effort required grows too.

[rytill]

Use "capability" instead of "intelligence" then. Defined as "ability to solve any problem dwighttk has ever dreamt up."

There's pretty much no reason to believe capability peaks roughly above the brightest human.

Our brains aren't yet even integrated with hardware-optimized algorithm solvers on which to offload minimax or tree search problems, or solve simple game-theoretic situations, or any number of things a computer system is much better and faster at than a human.

It's just another one of those things that you can believe if you want to not spend time worrying about the ethics problem.