[fwipsy]

I remember thinking exactly the same thing around 5-7 years ago, in the GPT-2/GPT-3 era. "Oh sure they can produce semi-coherent output, but truly intelligent behavior is still far away. This isn't science fiction, they're just falling prey to Pascal's Mugging same as my religious friends did." Now I'm not so sure. I give the AI safety subculture as a whole a lot of credit for putting it on my radar back when it was otherwise still science fiction. I don't know if they're right about what comes next, but I think their case deserves to be evaluated on its merits, rather than assumed to be the result of psychological flaws.

> Yall, with your incredible wealth and resources you could do real good in this world and make society better, healthier, better educated, and the whole world more equal, just, and reduce the desperation and suffering. Reject the false and self-serving narratives that empathy doesn't matter, that altruism isn't "effective". You can change a person's whole life in a moment.

Confused at who this is directed towards. I'm fairly certain that the article was written by people who (at some point) identified as effective altruists, most of whom would enthusiastically agree with this. This community didn't start as AI researchers and later choose effective altruism; they were effective altruists who chose AI safety research as the most effective way to improve the world. Given that you apparently share their goals (a better world,) isn't it worth at least hearing them out on their methods?

[romanhounds]

>Given that you apparently share their goals (a better world,) isn't it worth at least hearing them out on their methods?

Their methods are about convincing others that things that enrich and empower themselves at the expense of others is "improving the world". This isn't the stance of serious people who want to improve the world.

[fwipsy]

Please don't confuse AI researchers and AI safety researchers (although there is some overlap.) That's like confusing the people from the Manhattan project with the people who protested 3 Mile Island, because they're both focused on nuclear technology. One effect of EA is that every AI researcher says they're trying to make the world better. Some of them are full of shit, but not all.

I'm fairly certain the authors would be happy to see AI shut down indefinitely. They just don't believe that the coordination problem is solvable. This is their best attempt to come up with something workable in the real world, or at least get people started thinking about it.

[nradov]

Right, the so-called "AI safety researchers" are worthless clowns and grifters with nothing of value to contribute. We would all be better off if they spent their time on something productive like writing entertaining sci-fi stories. They certainly seem to have the rich imagination necessary for that occupation.

[fwipsy]

Now this is an unserious comment, just throwing insults without explaining your reasoning at all. Do you believe that AI itself is worthless? Or that it's just incapable of causing harm? Do you resent the safeguards placed on AI because they are personally inconvenient to you? When introducing a new technology at this scale, don't you think it's worth looking before leaping?

[nradov]

So called "AI safety researchers" are themselves unserious so of course I don't take them seriously. LLMs are extremely useful, I use them constantly for a wide variety of tasks. LLMs don't cause harm: humans cause harm. There's no real need to intentionally cripple or restrict the models in the name of "safety"; that's just stupid and done only to prevent corporate embarrassment.

[esafak]

Is that your line with nuclear weapons too; that it's the humans cause harm, so you don't need to regulate them?

[nadir_ishiguro]

Sadly they're very bad writers. Have you actually read the stuff they produced? Horrible. Yudkosky himself wrote https://en.wikipedia.org/wiki/Harry_Potter_and_the_Methods_o...

It's so bad

[reasonableklout]

I'm confused, the authors of the website are arguing that the development of AI should be slowed down so that society can adapt. What do you think they are trying to convince others of?

[MaybiusStrip]

Which one of the authors of the website are you referring to?

[heyts]

> they were effective altruists who chose AI safety research as the most effective way to improve the world

This actually made me laugh, I’m sorry

[grey-area]

Effective altruism is also very attractive to manipulative sociopaths who want to maximise their power over others whilst appearing virtuous and hoarding wealth and power. Poster boy for this movement is the convicted fraudster SBF. I believe Altman is also a fan.

As to a better world or super intelligence, I’ll believe it may be possible when I see some signs of intelligence from what people are calling AI, instead of plausible text and image generation based on a very large corpus.

[0xDEAFBEAD]

>I believe Altman is also a fan.

Altman says he thinks it is an "incredibly flawed movement"

https://x.com/sama/status/1593046526284410880

The dislike is mutual. Here's a long video takedown of Sam from a major EA org:

https://www.youtube.com/watch?v=_eYTkvZqbnQ

[HDThoreaun]

And what movement isn’t attractive to manipulative sociopaths who want to maximize their power? That’s unavoidable when dealing with humans.

[notahacker]

An important adjacent point is the same people that are insisting that only their research can stop intelligent manipulative computers from controlling the human race are also some of the few people who believed that OpenAI was a philanthropic endeavour and that Sam Bankman Fried was trustworthy...

[fwipsy]

I'm not seeing any claims to exclusivity. I'm seeing people actively encouraging more people to enter the field of AI safety/security and openness to robust debate, etc.

I agree that there seems to be a huge problem with value drift--either people who used to research AI safety, pivoting to building AI (looking at Anthropic) or people who only ever paid lip service to safety (I tend to put Musk and Altman in this category.) These people need to be held accountable, but it doesn't mean every AI safety researcher ever was a stooge or a fraud.

[notahacker]

Ok, fair enough theyre not entirely opposed to outsiders setting up research institutes like theirs and enjoy debate (or the optics of debate) more than most, and I don't think most of them are insincere. But the point wasn't that they were exclusionary, but more that whilst priding themselves on their supposedly superior ability to predict the future and "align" AIs, they somehow missed the value drift and human non-alignment with their values basically any outsider saw coming...

[HDThoreaun]

OpenAI now controls one of if not the largest philanthropic endowments on the planet. They are still a philanthropic endeavor, though I agree the non profit bait and switch as well as the Altman board situation were tragedies.

[notahacker]

I mean, they're also gearing up for one of the largest IPOs in history to generate unprecedented amounts of wealth for themselves, and the nonprofit foundation's focus is community engagement with OpenAI products. Goldman Sachs has a large philanthropic endowment but I wouldn't confuse that with their priorities being philanthropy.

The point was I think pretty much everyone else saw the bait and switch coming...

[grey-area]

Well I think there is a definite pattern to sociopaths using a movement that seems externally good and has many true believers - like say an open foundation to promote AI, or effective altruism. Then twisting it to their own ends to gain power, influence and wealth.

We should therefore be very skeptical of people claiming altruistic motives while requiring that they be in control.

[danbruc]

I remember thinking exactly the same thing around 5-7 years ago, in the GPT-2/GPT-3 era.

ChatGPT was announced three and a half years ago, 30-11-2022.

[semiquaver]

Do you think that ChatGPT is when people on hacker news first became aware of the GPT series of models?

https://hn.algolia.com/?dateEnd=1594339200&dateRange=custom&...

[IshKebab]

And GPT-3 was released 6 years ago in 2020... What's your point?