[derefr]

A target of exactly-human intelligence is potentially useful for automating All The Jobs and entering a post-scarcity era without also starting an AI singularity.

Though many approaches to “exactly-human intelligence” would just result in humans that happen to be robots, and we probably don’t want that, especially if the point is to make them do all our work for us.

I don’t know how to refer specifically to the alternative, though—an intelligence that can solve “human-hard” problems, without needing anything like sentience or self-determined goals/desires that would push it toward wanting things orthogonal to its designed purpose. Human-adjacent AGI? Human-competitive Tool AI?

[dwc]

Much of your response seems to orbit around the utility function, which is the devilishly tricky part from the start. Getting that part right is much more important than a specific level of intelligence.