[andai]

>Is superintelligence just a memetic hazard? [Overblown fear by smart people who are too easily convinced.]

Well we can do the wager. If it's a nothingburger, then the worst case scenario is that we approached AI too cautiously. (Ha. What are the odds of that?)

If it's not a nothingburger, then we all die, unless the whole world agrees on the correct course of action in advance and coordinates perfectly. Hmm.

Well, maybe we don't all die, but the world is irreversibly transformed into something incomprehensible and repulsive.

Although, I don't really think we needed AI's help for that one. We should probably figure out how to align ourselves before we try to preach to the next species. I'm not exactly holding my breath though :/

[OkayPhysicist]

That's just Pascal's wager with a secular/cyberpunk reskin, and has all the same trappings. Why aren't you a monk?

[andai]

Yeah. Turns out game theory is useful in games.

I don't understand the monk question though.

[OkayPhysicist]

If sci-fi Pascal's wager applies, so does the original. Anybody who believes Pascal's wager should be dedicating themselves wildly to religion, in the chance that it saves from from eternal damnation. Hence, becoming a monk.

[andai]

Oh, I'm actually reverting much of the subconscious Augustinian programming (most of what we call secular ethics) to the less busted Orthodox versions, but I think that is for unrelated reasons.

[dweinus]

Ah yes, Pascal's mugging.