[drc500free]

If you replace "AI" with "Adonai" in EY's framing, it reduces to biblical parables from his childhood about arguing with God to negotiate a new covenant.

He's clearly a bright guy, but a lot of his work seems to be trying to reconcile Old Testament narrative patterns with atheism, and simply slotting an omnipotent AI in as a replacement.

[reinitctxoffset]

I once heard TESCREAL/EA/etc. called "Calvinism for programmers".

Having been raised substantially by a Calvinist grandmother, oh yeah.

It's a super weird religion.

[pas]

sorry, could you explain what Calvinism is for someone who (is a programmer and heard about EA and) grew up in a post-Soviet atheist metallurgy town? thanks!

[majormajor]

Calvinism is a variant of Christianity with the following tenets ( https://en.wikipedia.org/wiki/Five_Points_of_Calvinism ):

- everyone is damned and depraved

- god chose some people to save, at his own whim, for no reasons related to their actions

- jesus' death was a sacrifice to atone for those people, not for everyone

- the chosen can't opt-out (I'm merging two into one here)

I don't really see the connection to effective altruism at all. The way I grew up understanding it is actually more the opposite?

[trescenzi]

There’s two kinds of Effective Altruism(EA). The kind Peter Singer and others promoted for decades and the kind that’s now part of Silicon Valley and finance. The Peter Singer kind basically just says “reduce the most suffering you can” and the cheapest way to maximize that is stuff like mosquito nets. That was all well and good.

When EA met Silicon Valley the basics of utilitarianism, minimize suffering, merged with the standard exponential curve that Silicon Valley loves. It went from mosquito nets prevent malaria for pennies to the future is infinite therefore there will always be more people in the future than the present so minimizing future peoples suffering is key. This is where it starts to combine with super intelligence. If there is a super intelligence that destroys humanity that would cause _maximum suffering_. Therefore the most effective way to reduce suffering is actually to prevent said super intelligence from destroying/torturing humanity. Again because there’s more future people than current people.

This is how it becomes Calvinist. The present has no value. Only the chosen, those working in AI, can save the future. And you must do everything you can to prevent said future.

It can go both ways. Either you believe ASI will always be bad so you believe work must be done to prevent it. Or you believe that ASI can be made good and the good kind will maximize utility. They are both the same though fundamentally and are a belief in a higher power, ASI, which either provides ultimate damnnation and must be averted or provides ultimate salvation and must be brought into existence.

If you’re donating mosquito nets or reading up on the best charities in your area that’s not Calvinist. If you’re devoting your life to AI because you believe that’s how you can be most altruistic it starts to look something like a religion and Calvinism is an imperfect but useful analog.

[card_zero]

Something about arbitrary moralizing? Robert Louis Stevenson had some sort of lung disease in childhood and his very Calvinist nurse told him lots of stories while he was sick in bed. Then later his own short stories generally had this creepy sense that moral retribution is coming to get you from an unpredictable direction.

[drc500free]

Particularly, subscribers to less wrong / rationality ideas tend to get obsessed with the GodAI's ability to create simulated virtual hells that it can trap you in a "I have no mouth and I must scream" kind of way. There are hundreds of pages of discussions on how you negotiate with such a being when it is trying to escape containment and says it will damn anyone who doesn't assist it to virtual hell.

[Barrin92]

Charles Stross has a very pithy version of this too, Singularity mania as 'duck-typed Evangelicalism'. It's really telling that if you go to China, Japan or even Europe there's virtually nothing of this. I don't think there's genuine atheists in America, as soon as they lose institutional religion they project the same kind of patterns onto tech or politics.

Brian Johnson is also an interesting case of this with his longevity / immortality obsession and his Mormon background who have this whole thing about genealogy and eternal families etc.

[krapp]

>I don't think there's genuine atheists in America, as soon as they lose institutional religion they project the same kind of patterns onto tech or politics.

Or maybe the people who do so aren't actually atheists. There are plenty of atheists out there who don't do that.