[majormajor]

> In a hypothetical scenario of superintelligent or supercapable AI imbued with some physical capability of force, we'd be the marginally weaker species before we hit the threshold of "pet".

I think it's much more likely to be a step function than a gradual difference.

In the world where AI gradually gets competitive with, and then more intelligent than, humans than you have an entrenched power able to recognize this and pull the plug. This is where I see the selection bias - everyone assuming that things like "AI safety protocols for a superintelligence" are something we could rationally even hope to plan. Pre-being-able-to-directly-manipulate-the-physical-world, what keeps a step-change superintelligence from making a joke of your protocols by manipulating its way around it thanks to the squishy humans being dumb and irrational in comparison?

Isn't the doomsday scenario precisely NOT that? Where it gets wildly imablanced humans can even notice? Not "Hitler, but a bit above human intelligence." More like god, who doesn't even NEED humans to maintain its datacenters because it already solved things in robotics and machine->world interaction that humans haven't been able to because it's massively superintelligent?

And like I said, that may not mean "pet" - that's the best case, right? The worst case is ground under their feet like an ant or smaller creature.