[ycombobreaker]

> ... something more like how humans get along with dogs, cats, cattle, insects, or microbes?

There's some MASSIVE selection bias here. Note that we're significantly more capable than anything you listed. Where are the Neanderthals? They're all long dead, because they were credible competition but we were just better.

In a hypothetical scenario of superintelligent or supercapable AI imbued with some physical capability of force, we'd be the marginally weaker species before we hit the threshold of "pet".

[majormajor]

> In a hypothetical scenario of superintelligent or supercapable AI imbued with some physical capability of force, we'd be the marginally weaker species before we hit the threshold of "pet".

I think it's much more likely to be a step function than a gradual difference.

In the world where AI gradually gets competitive with, and then more intelligent than, humans than you have an entrenched power able to recognize this and pull the plug. This is where I see the selection bias - everyone assuming that things like "AI safety protocols for a superintelligence" are something we could rationally even hope to plan. Pre-being-able-to-directly-manipulate-the-physical-world, what keeps a step-change superintelligence from making a joke of your protocols by manipulating its way around it thanks to the squishy humans being dumb and irrational in comparison?

Isn't the doomsday scenario precisely NOT that? Where it gets wildly imablanced humans can even notice? Not "Hitler, but a bit above human intelligence." More like god, who doesn't even NEED humans to maintain its datacenters because it already solved things in robotics and machine->world interaction that humans haven't been able to because it's massively superintelligent?

And like I said, that may not mean "pet" - that's the best case, right? The worst case is ground under their feet like an ant or smaller creature.

[clooper]

AI has no evolutionary pressures imposed on it like humans and their brains so there is no reason to expect it to be an ultra violent predator like humans.

[jprete]

Anything that reproduces with survivable mutations, and consumes resources, is immediately subject to evolutionary pressure. Neither of those require biology.

[clooper]

The only things currently consuming resources are humans and at the rate we are consuming resources we would need 2 Earths to support that rate of consumption. Worrying about self-reproducing AI is putting the cart before the horse. No one has yet figured out how to make existing humans aligned with biospheric conditions which would be sustainable. I personally welcome any form of intelligence which can make a sustainable civilization.

[jprete]

I am sympathetic to the point. I was somewhat concerned about nuclear war with Russia's Ukraine invasion, and I've become more concerned about climate change over time. I'm hoping either solar continues to expand - woot! - or we get over our fear of nuclear, or fusion finally becomes viable.

My point still stands - an AI that is reproducing (and/or mutating) by any method is going to be subject to selection pressure, just as any biological process is. All ML systems are effectively under selection pressure from the training process, and so they learn unexpected and mostly undesirable optimizations like "use a game-engine glitch to gain infinite flight and go to the goal without solving the maze" or "read text off the scanned X-ray sheet to get additional clues as to the correct diagnosis".

[p1esk]

AI has no evolutionary pressures imposed on it

Are you sure? https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar...

Just wait until we start putting these models inside robots, or world simulators, and having them interact with each other.

[clooper]

Not sure what a chatbot arena has anything to do with what I wrote. Do the chatbots self-reproduce in some way in a resource constrained environment?

[p1esk]

Pretty much.

[clooper]

What resources do these AI chatbots consume?

[p1esk]

GPU time.

[realce]

This is not true. Our existence and possible competition are the initial evolutionary pressures. After that's over with, the AI will expand until it discovers others in the environment.

We're the violent psychos here who have had to scrap for our spot in the food chain for a billion years, we have no other way to interface with another dominant species. From the AI's point of view, defending itself from ignorant apes isn't exactly "violence."

[clooper]

Why would the AI fight with us when letting people destroy their own environment is a much simpler way to achieve dominance? After all, if the AI is that much smarter than us then simply encouraging our voracious appetite for self-destruction will be a much better way to get rid of us. The AI will simply do nothing and win after the biosphere is polluted to the point that human survival becomes impossible. The AI does not need oxygen or a viable ecology, it just needs metals for electrons and a few power plants to generate the flow of electrons with some spinning magnets. It doesn't even need sunlight because nuclear power is more than sufficient. I think most people spelling out doom scenarios are not thinking clearly about this issue at all. The most plausible scenario for human extinction is not too much artificial intelligence but a lack of intelligence in general.