[dougmwne]

I suppose it depends on your premise. Is your AGI like a GPT-20, something that takes human input and intention and completes the mental labor like having a team of 1000 expert humans? Or is it more like a superintelligence that has its own will and goals that are potentially beyond human capability to understand?

If you’re talking a superintelligence, then forget it. The human race will have been surpassed and if we are kept around it will be like pets.

If it’s human level AGI under firm human control, then the value of intellectual work will approach zero, but the value of humans will, direction and discernment will be multiplied many times over. At that point it would be time to head for the stars.

[squigz]

> If it’s human level AGI under firm human control

This just sounds like slavery with extra steps

[dougmwne]

There’s certainly a kind of moral hazard here, and one that humanity has a terrible track record with. If an AGI has human-level capability, but is unconscious and non-subjective, then it’s just a really incredible tool. It would not be the first time we othered an intelligent, subjective being though.