[estearum]

Basically all critiques of Anthropic's policy moves on these topics boil down to people not believing the fundamental concerns are real, and often then going a step further to conclude that Anthropic doesn't actually believe their concerns either.

If you believe Anthropic believes what they say they do, all of it makes sense.

[caconym_]

Even if you believe the concerns have merit, it's hard not to be cynical about people (e.g. Anthropic leadership) paying lip service to those concerns while so obviously leveraging their power and wealth (which depend, by the way, on accelerating the world toward those hypothetical "concerning" scenarios as fast as possible) to position themselves such that they will become unimaginably richer if things go their way, and will also come out on top pretty much no matter what happens.

It's like a prisoner's dilemma where one party is loudly lecturing the other about the obvious benefits of cooperation while also obviously working on defecting. They want to have their cake and eat it too. Maybe they really are the pure-of-heart Chosen Ones destined to lead us around the great filter, but I don't see why I should believe that's the case when their behavior is just as easily explained as maneuvering toward being the winner who takes it all.

[estearum]

> (which depend, by the way, on accelerating the world toward those hypothetical "concerning" scenarios as fast as possible)

Yes, this dynamic is exactly the one that anyone who's concerned about AI is concerned about. I don't know why you state this as if it's evidence against the concerns lol. Someone being concerned about the incentives of a situation doesn't de facto make them immune to those incentives, obviously.

The implication that someone who's concerned about an arms race dynamic could simply opt out of the system that produces that dynamic is simply confused about what arms race dynamics are. The entire point is that it's a trap, and it's a trap even if you know it's a trap, and even if you don't like that it's a trap. There's nothing dishonest or hypocritical about being in the trap: it is literally a trap –– that is what it does and why it is bad!

I'm confused by these comments that imply people believe Dario et al are "pure-of-heart Chosen Ones destined to lead us around the great filter." Who? I've never seen it. And any AI-doomer is probably of the opinion that the entire question of Dario's or anyone else's personal moral character is 99% irrelevant. Because, again, it's a trap. The dynamics at play are so much larger than whether someone irks people for their lecturing tone. I would much rather give my money to Dario, who seems like a generally good person, versus Sama, who seems like a complete snake, but I'm under no illusions that doing so changes the fundamental dynamics that are steering us to AI doom. I doubt anyone does.

And yes, obviously they are angling toward being the winner who takes it all. That is literally the trap. If you believe what they believe, yelling "let's cooperate!" while hurdling towards the finish line and tripping your competitors is the only reasonable thing to do. That is the problem.

[senordevnyc]

This is an excellent comment, and I agree. I do think that there’s also evidence that Altman’s behavior can also be explained as a person who is naturally manipulative also being stuck in the trap and responding to incentives. But not necessarily a snake just in it for himself. The thing I keep coming back to about Altman: he doesn’t have any equity in OpenAI. And he definitely could have if he’d wanted. It’s hard for me to square that with the idea of him being greedy and self-interested.

[jcgrillo]

But the things they say they believe are insane and totally unmoored from physical, societal, and economic reality. If they actually believe those things they're untrustworthy because they're delusional. If they don't, they're untrustworthy because they're fraudulent. Either way it's not good..

[reducesuffering]

They're not. They're in the eye of the storm and see what's going on the clearest. They were ahead of the curve to be where they're at now, and they're still ahead of the curve for where we're going. All the other heads of labs like Sam Altman and Demis have been saying the same thing since 2015-2016 way before any of this "marketing" would ever have been at play.

[jcgrillo]

There's a simpler explanation that fits the data better: they're lying.

Generally, in the past when tech companies have made outlandish claims that were not backed by evidence, they're later found out to have lied. This is an ancient pattern going back to the dotcom era and before, but for recent examples you need only look back a few years to the web3 era. If they're not lying, they can show it by producing the results they claim. Until then, they're probably just lying.

[estearum]

What data does "they're lying" fit better than "they're earnest?"

> If they're not lying, they can show it by producing the results they claim. Until then, they're probably just lying

Brilliant framework: Anyone making claims about the future is not just speculating, not just wrong, but they are lying.

[jcgrillo]

> What data does "they're lying" fit better than "they're earnest?"

Claim: GPT-4 is a PhD level expert.

Claim: LLMs "reason"

Claim: LLMs "understand"

Claim: AI will automate jobs away

None of these are "earnest" claims for a company to make (whatever that means). They're vacuous bullshit, and false. It's a bunch of wordsmithing papering over results that are either hidden from view or just unimpressive. Even if the person saying that shit believes it, that's not super relevant--if a company is advertising these capabilities then... just... doesn't do it, that's lying. Corporate statements aren't unilaterally determined by fallible individuals, they're reviewed, crafted products. They can be fairly critiqued as such.

> Brilliant framework: Anyone making claims about the future is not just speculating, not just wrong, but they are lying.

Not just anyone, companies in particular. If a company tells you it's building something to replace jobs, and then it doesn't do it, that company lied.

[estearum]

Lol, I can name 3 specific jobs within my own company (of <10 people) that AI prevents me from having to hire for. They've been automated away.

My company itself (possible only with AI) does the work of at least several dozen people across my hundred customers or so. Those jobs are now automated away.

Does that mean you're lying, or just overly confident (and wrong) in your speculations?

FWIW, I wouldn't put Sam Altman in the category of "earnest." I'm not sure if you just aren't aware that Anthropic and OpenAI are different companies, or if you're arguing dishonestly by trying to put sama quotes in here? But weird move in either case!

[shimman]

What are you referring to? The cult belief that they are ushering in a machine god or that they strictly care about making as much money as humanely possibly while ignoring the absolutely destructive impacts these companies have had on society?

IMO they are using the cult messaging to distract the public so they take out all the oxygen in the room regarding people that care about the immediate impacts (climate exacerbation, ease of scamming, degrading job prospects, increasing income inequality).

Whenever real concerns are brought up against these companies they are always ignored while claiming the real concern is the fantasy of a machine god turning into skynet.

[estearum]

"Why don't they just not participate in the arms race?!" - guy who's never heard of arms races

If they believe they're creating "a machine god" and that it's better it's their machine god than someone else's (which, given the other contenders, I tend to agree with), then all the corollaries you mention are mostly irrelevant.

Whether you believe they're creating a machine god is irrelevant. They believe that they are. It would be helpful if you could create an actually good argument for why they cannot or are not creating a machine god, but it turns out there are no good arguments for why it's impossible to do so. And so... they shall try.

[paulhebert]

A lot of people would prefer nuclear deproliferation over building more nukes.

Arms races always work out great for arms dealers. Less so for the average Joe.

[shimman]

Oh okay, they're all just legit crazy and are allowed to poison the environment, murder teenagers, and ruin the material lives of millions for fantasy level delusions.

Good to know.