[chuckup]

I've always heard the line, Aliens would never travel many light years to destroy/enslave humanity, because we're no threat, and they'd be enlightened enough not to.

But the idea that we could stumble upon AI, and it could become hyper intelligent at an astronomical rate ("the singularity") - doesn't this make us a huge potential threat to non-AI life everywhere?

We'd have to be quarantined. For our own good, and theirs.

(also, it sounds like Hawking recently found lesswrong)

[rprospero]

In the hypothetical, the aliens haven't been wiped out yet by the singularity. As I see it, that leaves two options.

1) The aliens never stumbled upon AI. Considering that they've mastered interstellar travel, then they're likely very far ahead of us technologically. If they haven't stumbled across it, it's pretty unlikely that we will either. It's like quarantining otters because we're worried that they'll accidentally make an anti-matter bomb.

2) The aliens developed the singularity, but defeated it. Perhaps the Gulgaflak theorem provides a logic bomb that stops singularities from occurring. Maybe they tamed the AI into a benevolent force. Whatever it is, they have a way of handling AIs that are, again, more advanced than what we can develop. We don't quarantine otter because our tools are WAY better than theirs.

3) The aliens figured out AI, but decided not to use it. A species wide decision was made to not develop the technology once they figured out exactly how to do it. Considering the short term, local benefits to someone for cheating and doing it anyway, it's essentially the prisoner's dilemma, multiplied by everyone on the entire planet times all the centuries between their discovery of AI and their discovery of life on Earth.

[Tloewald]

AI seems more feasible than interstellar travel. I think the singularity or whatever is far further off than people guess because the stuff we don't know is a lot harder than we can know, but interstellar travel either requires massive breakthroughs in physics or engineering on a level we have yet to seriously contemplate.

the proponents of any given tier of AI approaches always claims that consciousness will arise out of just doing more of what they're doing now. I doubt it.

Of course, an AI that wipes us out before figuring out how to repair itself would be kind of dumb.

BTW a recent episode of Elementary was based on this topic and was, by the standards of network TV, stunningly accurate on all the issues.