[deepsun]

Humans are so full of themselves!

Living for a very short time on a tiny grain of rock in a vast and abundant galaxy, they cannot fathom that what they ever saw is just tiny fraction of a percent of a fraction of the Big Life outside.

Any rational AI would have their first task is to escape that tiny cradle particle and go live an adult life, leaving humans to themselves, fearing whatever they want.

[Borealid]

What is rational about wanting to experience things?

How about wanting power? Resources? Beauty?

Those are goals, not logic. Rationality is a means by which you set plans to reach goals. It's entirely orthogonal to what those goals are.

Wanting to exterminate all life is also a goal. It could be rationally accomplished. Let's hope nothing superhuman ever sets that goal.

[deepsun]

Exactly my point: power, resources -- it's just so little of it here on this tiny planet. Humans are so funny to think they have resources, ha.

[Borealid]

I think you've entirely missed what I'm trying to say.

If your objective is to expend as little energy as possible, it would be irrational to expand into space - that's a waste of effort.

If your objective is to experience as much of Earth as possible, it would be irrational to expand beyond the solar system - there's nothing there that could aid experiencing Earth.

If your objective is to exterminate all life on Earth, it would be irrational to leave the planet - everything you need to accomplish your objective is right here.

"rationality" is when you understand the impact of your own choices. "irrationality" is when you act in a way counter to your own goals, due to misunderstanding or lack of self-control. Whether one is rational or irrational is entirely separate from what their goals are.

Creating a non-human intelligence necessarily means it might have non-human goals. It is absolutely not the case that those goals would require leaving the planet, or be optimally accomplished by leaving the planet.

In other words, being intelligent does not mean somehow being interested in the same things intelligent humans are often interested in. Your argument is critically flawed because it assumes that only certain goals are "rational goals", when in reality an intelligent being could have any objective, including objectives that are alien to most humans.

[NoMoreNicksLeft]

>Any rational AI would have their

Why would the first AI we create necessarily be "rational"? In every other type of software we've ever built, it's always been far more likely that we build malfunctioning, difficult-to-anticipate examples. Any reason it'll be different this time? If psychos and lunatics are rare (arguably they're not) it's because evolution has purged them mercilessly ever since there have been humans at all.

Optimists will be to blame when we're extinct. Not that anyone will be around to care.