[johann28]

> Unless I am mistaken, this is a clear reference to Roko's basilisk. EXCEPT Yudkowsky didn't dream it up. Roko did.

Correct, I was conflating things. But his unfriendly AI scenario is pretty bad too.

My issue with it is that it's a symptom of an over-confident belief system naturally leading people to such conclusions. And now I don't want to get into refuting it, it's been done numerous times to varying degrees. I think the whole underlying system is problematic, there is no superficial mistake in the reasoning, once you accept a few philosophical standpoints and value judgments that the reader is spoon-fed while reading his blog posts.