[HideousKojima]

EA is heavily rooted in the rationalist movement, which was kickstarted by Eliezer Yudkowsky. Yudkowsky wrote some HP fanfic called Harry Potter and the Methods of Rationality. Yudkowsky now runs some sort of foundation about averting "AI risk," and AI risk seems to be a disproportionately (and dare I say irrationally) big concern for rationalists and EA advocates.

[JamesBarney]

Just for context AI is ~10% of EA donations.

I'm not a huge AI risk guy, but it doesn't seem irrational to me. It doesn't seem unreasonable that you'll have a 5% chance of > than human IQ in the next 100 years. Assuming a 5% risk of it going rogue you end could end up in a pretty terrible situation.

[HideousKojima]

Given how seriously mamy rationalists take certain things like Roko's Basilisk I have a hard time imagining that AI risk charities use their funds effectively (the E in EA, after all). That's assuming I agree with your 5% estimate (I don't).

[ben_w]

I have literally never encountered someone who really takes Roko's Basilisk seriously.

Other than the person who came up with it, I've not even heard of such people.

I don't doubt they exist — I've met people, people are dumb and believe dumb stuff even when they're smart — but I've never once encountered that.

[HideousKojima]

>Other than the person who came up with it, I've not even heard of such people.

Yudkowsky took it seriously enough to ban it from discussion on LessWrong as it was an "information hazard," so I think you might be a bit naive about how seriously many rationalists took/take it.

[ben_w]

“an incredibly costly way for me to learn that deleting a stupid idea is treated by people as if you had literally said out loud that you believe it, but Roko being right was never something I endorsed, nor stated.” - Yudkowsky

https://web.archive.org/web/20140626092902/http://www.reddit...

So no, not naïve.

[HideousKojima]

He didn't just delete a post about it, he outright banned discussion of it as an information hazard, so you'll forgive me if I don't believe him.

[JamesBarney]

I spend a lot of time in EA adjacent spaces and I've read the Basilisk there once, but I've seen it in several popular magazines reporting about EA 4 times. I really don't think it's a top concern at Givewell.

I've never seen seen a survey of AI experts where less than 50% thought we wouldn't have AGI by 2100. Why are you so confident that AI is an order of magnitude less likely in the next 100 years then the vast majority of experts in the field?

https://nickbostrom.com/papers/survey.pdf

[cauthon]

Are they AI experts? The authors of the paper you linked are philosophers, not CS. Yudkowsky doesn’t have even a high school education, and I’ve never seen any programming or ML work by him.

Frankly, the “research” in this area all appears to be science fiction that takes itself too seriously.

[JamesBarney]

Two of the groups they surveyed were AI researchers, it break it does by those groups, and the majority of those groups think AGI will happen in 100 years.

1. EETN: Members of the Greek Association for Artificial Intelligence (EETN), a professional organization of Greek published researchers in the field, in April 2013. Ca. 250 members. 2. TOP100: The 100 ‘Top authors in artificial intelligence’ by ‘citation’ in ‘all years’ according to Microsoft Academic Search(http://academic.research.microsoft.com) in May 2013. We reduced the list to living authors, added as many as necessary to get back to 100

[foobiekr]

Oh, MIRI. I didn’t know about the Harry Potter aspect. Everything I have seen out of MIRI, however, is completely ridiculous. So that makes sense.

[gnramires]

I consider myself close to rationality and EA, so maybe I have to explain myself here.

First, I call myself rationalist because of the culture around the blog SlateStarCodex (and r/<same name>[1]), by Scott Alexander (now at AstralCodexTen[2] for interesting reasons), and a bit less lesswrong.com , and also a few sources like Julia Galef (which is popular for the Scout Mindset[3]) and some people around this movement. The ideas are fairly tame and simple around encouraging clear thinking and avoiding biases -- but they go fairly deep into promoting an understanding how our brains and minds work (frequently discussed in SSC). I am partly defined by this now, I learned a lot about myself and in some ways improved my writing and why not thinking (yes, thinking more effective in general terms is not as well defined as it seems, and not nearly as effective as it seems either -- but in a certain sense I think it's good). I have not went through EY's extensive work to critique or embrace it. I think this movement is partly a response to the very damaging ideological war environment of the late 2010s and still ongoing that creates divided societies and not bringing us closer to a move charitable, loving, understanding, good future (instead lost in endless culture wars -- that in a way became very real wars with millions of affected). EA is a nice sort of consensus result from (part?) of the community as how one should approach charity and ethics (but none of the original EAs came from rationality I believe, they are largely academic philosophers from Oxford).

I'm an EA and I'm not sure where I stand on the AI debate. I personally have embraced the philosophy (although reluctantly, I try to follow Scout Mindset principles, which is I try to stay open to changing my mind), but I mostly give to GiveWell and to GiveDirectly. I also give to a few local charities in my quite poor country (written about here[4] -- I believe donating locally can be effective), volunteer and to many open source projects (some public seen here[5]). I made a commitment with myself to live frugally and give all that I can reasonably give, without unreasonable excesses (largely consumerism, luxury goods, things I don't need to work, etc.). I really believe in the potential of the movement, and rationality does help, because in the end it invites you to embrace the value of other lives, which I've become convinced with by many angles (from a metaphysical perspective, to a logical, social and economic ones): that's the foundation of my giving. Recently, Scott A. wrote that EAs really believe in the movement/philsophy, and the recently criticisms haven't really shaken my belief in the fundamentals (although I'm saddened by SBF and his statements, which I find deeply misguided and unethical, more here[6]). I also subscribe to Jane Goodall's (and Singer's) observation that we need to make the Head (reason) and the Heart (love and humanity) work in harmony to thrive as a society and as individuals -- most people have good will, but if you don't use reason and solid ethics to guide your good will we often fall short, sometimes tragically and spectacularly.

I believe other people matter, and I'm committed to trying to make the world a better place with whatever tools I find most appropriate (statistics, economics, social sciences, art, philosophy, technology, math ...): It's very simple, very robust, and very necessary for our collective future as well :)

(which is to say: Hack the Planet!!!)

[1] https://reddit.com/r/slatestarcodex

[2] https://astralcodexten.substack.com/

[3] https://www.youtube.com/watch?v=3MYEtQ5Zdn8

[4] https://www.reddit.com/r/EffectiveAltruism/comments/yrhl3g/l...

[5] https://liberapay.com/gustavonramires/ see also: https://www.reddit.com/r/EffectiveAltruism/comments/v7ma0d/w...

[6] https://mastodon.social/@gnramires/109337319931759374