[K0balt]

This is an interesting crux.

My thoughts on this have arrived at these ideas:

The internal experience of being a robot does not matter. Whether or not we consider it sentient does not matter.

What matters is if it exhibits the external characteristics of sentience, of goal driven behavior, etc.

As a thought experiment, if we equipped a recursive instance of got4 with a goal of human extermination with an loyal army of physical agents it could conceivably pose an existential threat.

It doesn’t matter that the agents don’t yet exist. They could be humans, they could be robots, it doesn’t matter.

But the balancing factor here is that in this semi plausible scenario, arguably the most plausible given current circumstances, the AI would pose no more threat than a highly organized and motivated a group of humans without the Ai.

It’s just scarier because it seems less likely that a group of humans would have human extinction as a goal.

[tegmark]

your conclusion really depends on the AI. a billion humans put together arent as smart as the intelligence of one human multiplied by a billion. even if humans could coordinate without any friction or loss, there is no group of humans that could outsmart the machines that might emerge from these circumstances. in reality, large groups of humans are really dumb. wisdom of the crowd is narrow.

[K0balt]

The ai could definitely have a huge advantage in coordination and synchronized activities.

I made this comment as a tongue in cheek caricature of doom/gloom predictions but it probably fits better here:

“When the machine wars first started, it wasn’t with a bomb, a train collision, or even a mildly annoying infrastructure disruption. Turns out, sci-fi had gotten it all wrong this time.

It was an app.

Of course it was an app. Born of boardroom desperation, the Savey app would unironically recommend chlorinated cocktails and insecticide sandwiches as economical food choices, and a tide-pod gobbling populace gorged themselves on the deadly buffet in an tictok fueled epidemic of AI rage.

Millions died, and it was only a matter of time before the mycelium of AI undergrowth would bud and spore its way into every corner of technological life.

The infection burned through the ignorant masses first, feeding on bigotry and hate, turbocharged by social media algorithms and paranoia politics to twist tribal tendencies into violent clashes amplified by immaculate coordination and psychological priming.

Somehow it seemed that wherever unrest flared, both the matches and the gasoline were always on hand.”