[beshrkayali]

This is hilariously true

> AI risk is string theory for computer programmers. It's fun to think about, interesting, and completely inaccessible to experiment given our current technology. You can build crystal palaces of thought, working from first principles, then climb up inside them and pull the ladder up behind you. People who can reach preposterous conclusions from a long chain of abstract reasoning, and feel confident in their truth, are the wrong people to be running a culture.

I understand how people running in the same scene fall into the echo chamber effect and get gulped into the cult, but why does everybody want to be a prophet?

[0xDEAFBEAD]

Theorizing about nuclear winter is somewhat similar, in the sense of being inaccessible to experiment. Does that mean we should disregard the possibility of nuclear winter?

[m4rtink]

As with using the Orion drive for launch and landing on Erth is possible, test nuclear winter is certainly possible. But as with the former, you won't be very popular among the survivors (if any).

[walrus01]

Being a prophet is probably great until you suddenly find yourself building a fortified compound in Waco, Texas and purchasing black market full auto machine guns.

[saltwatercowboy]

Thank you for contacting Gunmetal Ranch, a legitimate 501(c). If your call is related to the class-action "most dangerous game" settlement, please hold for a cowpoke.

[int3trap]

> but why does everybody want to be a prophet?

Its not everyone building crystal palaces in their mind, they're all building fortresses. And they can't be wrong in their fortress or it breaks their world view which they cannot accept.

[tim333]

AI is "completely inaccessible to experiment given our current technology"?

I think experiments have happened and even been mentioned on HN occasionally and effected RAM prices.

[angoragoats]

> AI is "completely inaccessible to experiment given our current technology"?

No, that’s not what the article/talk said. It said AI _risk_ is inaccessible (as the poster you’re replying to correctly includes in their quote). The point in the original talk is clear if you read it; that everyone coming up with their own pet theories for how superintelligent AI could destroy us as a species is building their theory on speculation on top of speculation.

Note that the talk is from 2016, before transformers and LLMs were invented. Though I would argue that the vast majority of it, including the parts we’re discussing here, are still completely valid.