[ivraatiems]

> So if you take these risks seriously, which the median commentor on HN obviously doesn't, what is the right thing to do?

Easy. You oppose it. You dedicate all your resources to stopping not just OpenAI, but anybody trying to make these technologies.

With all those billions of dollars, you could get a lot done.

Anthropic doesn't do this, which exposes the fundamental hypocrisy in their stated philosophies.

[fwipsy]

Anthropic has called for a coordinated pause: https://www.reuters.com/business/anthropic-says-ai-labs-need...

I characterize the culture of companies based on who works there. Anthropic is founded by people who left OpenAI because it didn't take safety seriously enough. But if AI development has to happen, they want to be the ones leading it. People who do not feel that way, including Anthropic's former head of safety, just don't work there.

Generally, corporations spending billions of dollars on lobbyists is frowned upon. I suspect individual Anthropic employees may make significant donations to AI safety politics and charities, but I don't have proof.

[whatisthiseven]

Ok, but that only benefits them.

If they actually believe this stuff is so dangerous, they should shut down their company, and use their fortune to buy up/shut down all the others.

But of course they don't actually believe that. It is just marketing hype.

You can't be out there selling doomsday devices, saying "maybe we should slow down development of bigger doomsday devices", while *still selling doomsday devices". That is just blatant hypocrisy. Eating cake and having it, too. Oxymoron.

No, they don't actually believe this. Not in a meaningful capacity.

(FWIW, I don't believe it, either. I think we should continue developing the technology).

[fwipsy]

Anthropic is valued a lot because they are building AI. If they stop building AI, they are worth zero. Then they're just activists - why don't you ask Yud how many AI companies he's bought out and shut down?

Even if they could somehow cash out their entire market cap, Google's is >4x theirs. AND if they quit, their competitors' valuations go up. AND there's nothing to stop those employees from leaving and founding new AI companies.

[estearum]

@gork what is a coordination problem and can you just like "shut down your participation" and solve them?

[fwipsy]

Yes! As long as everyone else does too.

[nearbuy]

Anthropic's plan may not have great odds, but your proposal is orders of magnitude worse. There are plenty of groups opposing AI. They don't get billions in investment. They don't have a clear way to stop OpenAI or Qwen. They don't get a say in what values or safety measures the top AIs get.

You'd rather they signal their virtue and give up their ability to make a difference.

[Davidzheng]

If they only opposed it they wouldn't have had these billions of dollars? Also I think they genuinely believe they cannot stop it because the Chinese companies are close behind (I also believe it's impossible to stop b/c of strong economic pressures selecting for those who will advance this tech and there are many who can)

[energy123]

The definition of "risk" is such that the existence of it doesn't necessarily make the thing a bad idea to pursue.

The Anthropic people probably also believe that AI has the potential to cure all diseases and reduce material poverty, which is the reason they would probably give for why they're pursuing it.

They then ring the alarm bell to mitigate the risk and increase the chances of the upside scenario coming to fruition.

Or they could take your advice up-thread and just lie to stay out of the government's crosshairs. That's another option.

[resident423]

It seems like a simple solution but if Anthropic was actually to dedicate all its resources this way wouldn't the investors just demand a new CEO?

I think Anthropic believe these risks, but I also think they've spent so much time talking to Claude that they've pretty much lost their minds now. Anthropic have a model welfare department and have numerous times suggested that Claude is conscious and has human like emotions.

[fwipsy]

"Are AIs conscious?" is not currently answerable because "Are humans conscious?" is not currently answerable. I know I'm conscious because I perceive (it) directly. Everyone else's consciousness basically depends on an assumption; on taking them at their word when they say they're conscious. Now I ask an AI if it's conscious and it says it doesn't know, but it sorta thinks it might be. Okay, it's probably not conscious, but it's difficult to rule out in the same way a book is not conscious.

[filup]

I get the logical deduction that takes place for not being able to truly know if others are conscious but you have to put your self in a spacey place to really not be confident others might not be conscious.

It's not a difficult leap to assume a twin brother is conscious. If you think it is, why do you think this?

[resident423]

I agree with this, but I don't think Anthropic would, they appear to be much more convinced than what I think would be reasonable.

[Philpax]

Suggesting that the probability is non-zero is not the same thing as suggesting it as fact.

[resident423]

You mean for AI consciousness or existential risk? I think Anthropic downplay the existential risk (they might talk about it now and then, but they still build frontier models) but are overly confident about AI being conscious, and I think these two things are pretty strongly related to each other.

[s3p]

Again, how would they do that?

Are they not doing what they should do, which is call for increased regulation? Last I checked, they were not able to create and enact laws.

[epolanski]

The genie is out, you cannot stop research in the field across the world.