[k8t]

Yes, I agree with the reasoning behind Pascal's Mugging. But Pascal's Mugging refers to things that have an astronomically low chance of happening — like 0.000000001. But is the chance that AI is dangerous that low? Nobody in the world knows for sure at this point, due to how far away superAI might be, and due to the uncertainties in implementation. Therefore, if we use Bayesian thinking and spread it out, I'm not really sure you could put it below 1% (I pulled this number out of thin air, but everybody at this point is doing the same).

[Analemma_]

There's no magic probability value at which the Pascal's Mugging argument suddenly "kicks in". It's all about the utilitarian tradeoff of, "given this low probability, am I devoting the right amount of resources to preventing this terrible event? Could they be better spent elsewhere? And is the fretting harmful in and of itself, wiping out the expected gain?" The talk is arguing that the answer is yes to both of those latter questions.