[kalkin]

That's not actually the argument that the AI Risk people use. It is a possible fallacy for sure but what people actually argue if you read Eliezer Yudkowsky, Nick Bostrom etc is that there is a high probability of building superhuman AI this century, and that by default the outcome is we all die. You can debate these premises but they aren't Pascal's Wager.

[notahacker]

The relevant probability isn't "probability of superhuman AI being developed this century" but the probability my marginal dollars donated to this specific research org will lead to positive developments in superhuman AI that would not have happened if the org did not receive my money. That one's a much higher bar to cross especially with EAs having satisfied themselves that other marginal dollars definitely are efficiently saving lives. Hence "longtermism", which is a quite explicit argument conceived within the EA movement to reconcile the low probability of stuff like a particular AI research paper saving any lives whatsoever by arguing the utilitarian maths still work if you can multiply that low probability by a large enough future of humanity safeguarded.

Yudkowsky's personal estimates of the probability of superhuman AI and the probability of his own research institution fixing it may have been more confident than most people's but he and his followers are no stranger to presenting Pascals' wager arguments to people who doubt approaches are feasible (even more so when it comes to stuff he isn't working on like cryonics), along with more explicitly Pascal-esque stuff like intertemporal bargaining with future AIs, simulations and infinite rewards/punishments...

[kalkin]

Depends on the question. I agree that the argument for a random person giving money to try to reduce AI risk seems dubious, and touched on this in another comment - https://news.ycombinator.com/item?id=32394429 - although I'm not up to speed on the "longtermism" debate as such.

But I think the argument for a policymaker or ML researcher to be worrying about this is at least better than Pascal's Wager (and remains so whether or not you buy any particular whiff-of-Pascal claims about simulations, "acausal trade", etc).

[notahacker]

Agree on both points. (There's a perfectly good reason for EAs with a computer science skillset to want to work in AI with a focus on safety; it's a well compensated industry where they can probably solve important real world problems more reliably than AI researchers less concerned about the ethical implications of what they do even without any sort of singularity, it just the movement seems to have invested a lot more in talking about longtermism than anything else lately, and Yudowksy's always had some very... individual lines of reasoning)

[dogecoinbase]

> Nick Bostrom etc

Someone who's primarily famous for an argument that would have been equally true and perfectly wrong at every past point in human history is someone whose opinion should be discounted via every available mechanism at your disposal.

[kalkin]

You are not doing yourself a service if you decide that "every available mechanism at your disposal" ought to include just completely misstating what the opinion actually is, as with the Pascal's Wager analogy.

[dogecoinbase]

Do feel free to elaborate on "what the opinion actually is" and how, specifically, I have misstated it.

[kalkin]

Did so in my comment to which you originally replied.

To be clear, I have no idea what argument you believe Bostrom is most famous for and hold no particular brief for him as a thinker. But, repeating myself, the AI risk argument that he makes in Superintelligence and that Yudkowsky etc make is one that assigns a high probability to disaster. It's not a secularized Pascal's Wager based on multiplying a low probability by dubious infinities.