[antonvs]

Ignoring the “observant Jew turned atheist” aspect, the rest of that description seems apt. A comparison against the features of religions is valid here, considering that one of the criticisms is that these people are following a religious playbook rather than a rational one.

We don’t enact public policy based on predictions of a biblical apocalypse, nor do we ban televisions or computers because groups like the Amish disapprove of them. The same reasoning should apply here.

[librexpr]

None of the comment is apt. You're seeing people you disagree with and assuming that if they disagree with you it must be because they're crazy, and then you use words like "cult" and "LARP" to try to dismiss them without engaging with their arguments. This does not lead to good discussion.

I think that unaligned and sufficiently smart AI could kill everyone because I looked at the arguments for and against and came to a conclusion. Presumably this is the same thing you did. I know my own mind better than you, and I'm telling you that your attempts to explain why I disagree with you are completely off the mark.

> We don’t enact public policy based on predictions of a biblical apocalypse, nor do we ban televisions or computers because groups like the Amish disapprove of them.

The reason we don't do that is because we evaluate their arguments and disagree with them. It has nothing to do with who they are or why they believe what they believe.

[antonvs]

I'm seeing the software equivalent of flat earthers and I'm treating their ideas with the respect they deserve.

> without engaging with their arguments. This does not lead to good discussion.

There is no useful discussion to be had with a doomsday cult that's more interested in an ungrounded fantasy it's indulging in than anything to do with the real world. The adults will continue doing what they're doing and the cultists will be ignored, and that's the way it should be.

> I think that unaligned and sufficiently smart AI could kill everyone because I looked at the arguments for and against and came to a conclusion.

In some possible world, sure. But to make that actionable today you have to construct a very shaky and unlikely series of conjectures. Public policy correctly isn't based on that sort of wild speculation, because there are an infinite number of possible worlds, and most of them will never come to pass.

Here are some questions for you: do you think GPT-5 could be the AI you're talking about? Do you think that something like that could happen in the next 6 months? Do you think that is in fact the biggest risk we face from AI, today?

The factual answer to those questions is: no, no, and no. If you think GPT has changed anything in this respect, you've been fooled by a chatbot that's simply using statistics to guess what it should say. It's not conscious, it doesn't have desires or goals other than to respond to the prompt that it's been given with a statistically appropriate result, and its next version is similarly not going to pose any of the sort of risks you're talking about.

There are, in fact, real AI risks that we should be paying attention to, but they revolve around how people and particularly large organizations will use it against other people, around its impact on jobs, and so on. If we engage with those risks and address them, we'll be in a better situation to prevent more speculative risks in future. But if we waste our time engaging with people who don't understand the technology or what they're talking about, discussing risks that don't exist, then AI will 100% certainly have a very negative impact, which has nothing to do with "superintelligent conscious AIs". Don't let yourself be distracted by scifi-fantasy nonsense from what's actually going on.

In fact, if you think a bit harder about what you're doing, you might realize that you're focusing on a purely technological risk to escape from having to focus on what humans do to each other. But that's the real "control problem".

[coldtea]

No reason to ignore even the first part though. Culture matters, and reaction to parental culture is very often a driving force...