[ianm218]

What I feel like is missing in the common discourse here is that Anthropic genuinely believes that AI poses an existential risk for humanity either in terms of literal survival or extreme mass surveillance, human disempowerment etc. So if you take these risks seriously, which the median commentor on HN obviously doesn't, what is the right thing to do?

I.e. OpenAI just went full evil corpo mode and went all in on the Leading the Future PAC [1] to try and prevent any kind of regulation.

I feel like there is a reasonable path where they might agree with OP that the government has "mostly gone insane" but also think that US getting its act together and leading the way on sane regulation will be key to getting to a good outcome with AI.

[1]. https://en.wikipedia.org/wiki/Leading_the_Future

[airstrike]

We do not know for a fact what they genuinely believe, and many of us have seen companies act in opposition to their stated goals so the burden of proof is on them.

It's healthy to suspect ulterior motives from them.

[ianm218]

Can you be more specific on what you mean - how would you prove what anyone believes about something short of reading minds and what would proof look like to you?

Here is Dario writing about AI safety in 2016 [1]. Dario and others in the Anthropic circle have long been associated [2] with the effective altruism movement, which whatever you think about them, they are very concerned about AI existential risks. Ronan Farrow & the New Yorker did a mega deep dive on Sam Altmans history of being dishonest and manipulative, and it credibly reports that Anthropic cofounders left due to not trusting leadership there with safety.

On the other hand we have decades long evidence of Altman and co. being dishonest and employing people like Chris Lehane [3]. I have no affiliation with Anthropic beyond being a user of their products to be clear but it feels like HN is going to end up on the wrong side of this one.

[1]. https://arxiv.org/abs/1606.06565 [2]. https://forum.effectivealtruism.org/posts/53Gc35vDLK2u5nBxP/... [3]. https://en.wikipedia.org/wiki/Chris_Lehane

[airstrike]

It's not about proving people are honest. It's about knowing that people either die a hero or live long enough to become the villain. Not because they want to, but because it's a natural journey given this competitive game of corporations.

OpenAI and Sam Altman being dishonest (or not) really has bearing on Anthropic. It's wholly unnecessary to argue about those other parties when discussing Anthropic or companies in general. Put simply, Alice being dishonest doesn't make Bob honest. But interestingly enough, OpenAI at first was also regarded as uncompromising stewards of "AI for everyone". It's in their name!

Speaking specifically about Anthropic, being concerned with AI existential risks is incredibly self-serving because guess who owns the very powerful AI that needs to be treated carefully and not allowed into the hands of other countries or competitors? All under the guise of being for the people. Even if a hundred people within the organization are doing this out of the goodness of their hearts, the company is not actually bound to that behavior and can renege on it at any point in time—just like OpenAI did.

Besides, Anthropic cofounders left OpenAI effectively forever ago. Just after GPT-2, which was pretty useless still. It was a whole different company. And the young are often naive but that naiveté eventually fades. Investors make demands, business realities sink in, and founders compromise.

This has happened time and time again. Or did you forget how Google Chrome started? Or how the shading and labeling of ads on Google has evolved over time? Or that "Do no evil" has been dropped?

The thing about my argument is that it doesn't require a purposeful nefarious motive. Corporations exist to maximize profits. When they act perfectly according top their stated goals, they make some decisions that are not in the interest of everyone else. It's important to remember that.

[ianm218]

> so the burden of proof is on them

I was mainly responding to this I know what you mean.

The reason I bring up OpenAI is because a lot of the discourse has boiled down to "both sides are the same" which I don't think is fair.

[NickNaraghi]

Super interesting. How does regulation affect IPO prospects of OAI and ANTH? What’s the timing of each?

[ivraatiems]

> So if you take these risks seriously, which the median commentor on HN obviously doesn't, what is the right thing to do?

Easy. You oppose it. You dedicate all your resources to stopping not just OpenAI, but anybody trying to make these technologies.

With all those billions of dollars, you could get a lot done.

Anthropic doesn't do this, which exposes the fundamental hypocrisy in their stated philosophies.

[fwipsy]

Anthropic has called for a coordinated pause: https://www.reuters.com/business/anthropic-says-ai-labs-need...

I characterize the culture of companies based on who works there. Anthropic is founded by people who left OpenAI because it didn't take safety seriously enough. But if AI development has to happen, they want to be the ones leading it. People who do not feel that way, including Anthropic's former head of safety, just don't work there.

Generally, corporations spending billions of dollars on lobbyists is frowned upon. I suspect individual Anthropic employees may make significant donations to AI safety politics and charities, but I don't have proof.

[whatisthiseven]

Ok, but that only benefits them.

If they actually believe this stuff is so dangerous, they should shut down their company, and use their fortune to buy up/shut down all the others.

But of course they don't actually believe that. It is just marketing hype.

You can't be out there selling doomsday devices, saying "maybe we should slow down development of bigger doomsday devices", while *still selling doomsday devices". That is just blatant hypocrisy. Eating cake and having it, too. Oxymoron.

No, they don't actually believe this. Not in a meaningful capacity.

(FWIW, I don't believe it, either. I think we should continue developing the technology).

[fwipsy]

Anthropic is valued a lot because they are building AI. If they stop building AI, they are worth zero. Then they're just activists - why don't you ask Yud how many AI companies he's bought out and shut down?

Even if they could somehow cash out their entire market cap, Google's is >4x theirs. AND if they quit, their competitors' valuations go up. AND there's nothing to stop those employees from leaving and founding new AI companies.

[estearum]

@gork what is a coordination problem and can you just like "shut down your participation" and solve them?

[fwipsy]

Yes! As long as everyone else does too.

[nearbuy]

Anthropic's plan may not have great odds, but your proposal is orders of magnitude worse. There are plenty of groups opposing AI. They don't get billions in investment. They don't have a clear way to stop OpenAI or Qwen. They don't get a say in what values or safety measures the top AIs get.

You'd rather they signal their virtue and give up their ability to make a difference.

[Davidzheng]

If they only opposed it they wouldn't have had these billions of dollars? Also I think they genuinely believe they cannot stop it because the Chinese companies are close behind (I also believe it's impossible to stop b/c of strong economic pressures selecting for those who will advance this tech and there are many who can)

[energy123]

The definition of "risk" is such that the existence of it doesn't necessarily make the thing a bad idea to pursue.

The Anthropic people probably also believe that AI has the potential to cure all diseases and reduce material poverty, which is the reason they would probably give for why they're pursuing it.

They then ring the alarm bell to mitigate the risk and increase the chances of the upside scenario coming to fruition.

Or they could take your advice up-thread and just lie to stay out of the government's crosshairs. That's another option.

[resident423]

It seems like a simple solution but if Anthropic was actually to dedicate all its resources this way wouldn't the investors just demand a new CEO?

I think Anthropic believe these risks, but I also think they've spent so much time talking to Claude that they've pretty much lost their minds now. Anthropic have a model welfare department and have numerous times suggested that Claude is conscious and has human like emotions.

[fwipsy]

"Are AIs conscious?" is not currently answerable because "Are humans conscious?" is not currently answerable. I know I'm conscious because I perceive (it) directly. Everyone else's consciousness basically depends on an assumption; on taking them at their word when they say they're conscious. Now I ask an AI if it's conscious and it says it doesn't know, but it sorta thinks it might be. Okay, it's probably not conscious, but it's difficult to rule out in the same way a book is not conscious.

[filup]

I get the logical deduction that takes place for not being able to truly know if others are conscious but you have to put your self in a spacey place to really not be confident others might not be conscious.

It's not a difficult leap to assume a twin brother is conscious. If you think it is, why do you think this?

[resident423]

I agree with this, but I don't think Anthropic would, they appear to be much more convinced than what I think would be reasonable.

[Philpax]

Suggesting that the probability is non-zero is not the same thing as suggesting it as fact.

[resident423]

You mean for AI consciousness or existential risk? I think Anthropic downplay the existential risk (they might talk about it now and then, but they still build frontier models) but are overly confident about AI being conscious, and I think these two things are pretty strongly related to each other.

[s3p]

Again, how would they do that?

Are they not doing what they should do, which is call for increased regulation? Last I checked, they were not able to create and enact laws.

[epolanski]

The genie is out, you cannot stop research in the field across the world.