[ajyoon]

I have never understood this line of thinking. Why would a company warn that their technology is dangerous if it wasn't? Looking at the ongoing Mythos export control issue, it's obviously not good for profits. If you assume AGI is possible, you must agree that it would be at the very least profoundly destabilizing. The companies are built around this assumption.

[anon373839]

> Looking at the ongoing Mythos export control issue, it's obviously not good for profits.

This is not obvious at all. Anthropic's biggest investor was the catalyst for this action in referring the "jailbreak" (if you can even call it that) to the government. Now, Anthropic is sitting around a table with government officials who are designing benchmarks that will determine what Anthropic's competitors will be allowed to build. At this point, I have no evidence to rule out the possibility that Anthropic's leadership purposefully sought this outcome.

[_aavaa_]

*why would they warn if it they didn’t believe it.

2 words: regulatory capture.

Ban Chinese models because they’re “bad”. (can’t have head to head competition).

Bad open weight model (can’t let people escape the subscription model).

[ajyoon]

People have been warning about the dangers of advanced AI since long before open weight competition was a consideration.

[_aavaa_]

Sure, but they’ve either been the kool-aid-drinking p-doomers, or academic discussions about giving them access to weapons.

Now we have real companies whose valuation depends on people using their services.

[britch]

> Why would a company warn that their technology is dangerous if it wasn't?

Because it makes their technology seem more important and powerful?

Do you usually believe 100% what a company says about itself?

> Mythos export control issue, it's obviously not good for profits

Who cares about profits? None of these companies make money like a business currently. It's all speculative valuation and influenced by the same hype/doom cycle

[SpicyLemonZest]

When people discuss the negative effects of climate change, do you think that boosts fossil fuel company valuations by making their products seem more important and powerful?

[bryan0]

I agree. There are 2 groups of people:

1. Those, like OP, that believe AI danger and disruption is all hype to boost valuations.

2. Those, like myself, that believe AI danger and disruption is very real and presents dilemmas (but also opportunities) for society, so we must tread carefully.

The first position is not logically consistent with reality. The danger is real: we have already seen hints about how this will impact everything from jobs to warfare to mental health. that danger does not increase valuation, in fact it does the opposite because of the need for government regulation.

[mrandish]

> in fact it does the opposite because of the need for government regulation.

For frontier leaders, regulation is a competitive moat. At trillion dollar scale, compliance teams, legal and lobbyists are a tiny fraction of opex. For emerging startups, early regulation of a new market is a substantial barrier to entry.

That said, I think Anthropic blundered into their current Fable mess unintentionally by underestimating just how much they pissed off the current administration by refusing to let the DoD use Ant's AI's for all the domestic spying and lethal combat ops they want.

[ekelsen]

I don't think the first position is steel manned by assuming those holding it actually believe there are no potential problems or ways to use the technology badly.

But that belief does not imply the doom rhetoric is necessary or the best way to approach those issues.

[JMiao]

that's an inaccurate characterization on #1. he's not saying ai risk is _all_ hype to boost valuations, but that it's misused to serve purposes other than warning.

[grebc]

They banned exporting bad cryptography algorithms/code 30 years ago too.

People just love to talk about anything that tickling their fancies.

It’s all a fugazi.

[ekelsen]

Because it makes them feel important because they are developing something so dangerous everyone else ought to listen to what they say.

It's about ego, not profits.

[chongli]

Why would a company warn that their technology is dangerous if it wasn't?

The whole point of the article is to answer this question, and here's the answer:

Because all the AI doom fear-mongering is driving sky-high valuations. The more the public panics, the more investors open their wallets.

[ajyoon]

Is there any evidence of this? It seems like a thing people say all the time, but completely unsubstantiated.

[chongli]

What else is there? The product as delivered today doesn't come anywhere near a justification for these valuations. All of it is built on an expectation of future capabilities, and that's where the Dario-penned doom papers come in.

[Barrin92]

>Why would a company warn that their technology is dangerous if it wasn't?

Because Americans are obsessed with eschatological narratives and action hero stories. I talked to a Palantir guy once who told me that he loves it when journalists describe his company as a James Bond villain because every single time the market cap goes up.

If they said that they're SAP with Call of Duty marketing they'd actually lose money. In other countries this strategy might not check out but in the US it's better to pretend to be a supervillain than to be boring

[prasadjoglekar]

Setting up for the eventual bailout. Remember - "we can't let banks fail or it'll be a depression"?

[SpicyLemonZest]

I remember a meme that spread to that effect, but it was similarly divorced from reality. Hundreds of banks failed during the financial crisis, including multiple which were very large. There were a few cases like Wachovia, where an unhealthy bank was required to sell itself in order to avoid a technical failure that would have impacted over 10 million Americans' access to their money. But this was still unpleasant for existing Wachovia shareholders.

[manoDev]

It’s the same narrative as “the communists are developing nukes so we need to develop first”.

[jmye]

> it's obviously not good for profits.

Weird to think profits matter half as much as ever-increasing valuations, driven by memetic bullshit. E.g. the entire point of the article you're commenting on.

[ajyoon]

Anthropic's valuation is driven by its revenue growth. Export controls harm this. Hype certainly plays a role, but it's simply not the case that revenue is not a major part of the picture.

[dartharva]

Regulatory capture and false AGI hyping, what else? You'd have to be willingly ignoring the writings on the wall if you still haven't identified it yet.

[mrandish]

> Why would a company warn that their technology is dangerous if it wasn't?

It helps to look at the history of the AI Alarmist, EA-adjacent mindset. They believe AI could be Dangerous (<--- capital D) if not responsibly managed by the right people (spoiler: themselves or politicians who listen to them). The capital D means more than just some economic and employment disruption, they're thinking much bigger, like "existential threat to humanity" big.

The most extreme alarmists are basically LARPing Terminator 2 and want to shut it down before SkyNet takes over but the more moderate alarmists believe AI is too potentially beneficial to walk away from. And they acknowledge other nations and various bad actors won't stop anyway, so they believe it's their sacred duty to move forward quickly albeit very responsibly. Yes, this leads to some conflicted reasoning and priorities, like they need to keep gaining access to gobs of capital to ensure they are the ones to reach AGI first, as that's the only way to ensure "god" is benevolent.

While I'm sure many of the top AI leadership are primarily (or purely) mercenary, I also think some, if not many, other execs and employees sincerely believe (or at least hope) they are "the good guys" playing a crucial role in an era of historical importance. Obviously, this has some heavy vibes as well as a being quite a buff to one's self-importance. They face each new day feeling the weight of bearing such a consequential role in shaping the future of humanity. Us mere mortals can only pray their wisdom and altruistic ethics match their humility. </s>

So... yeah, that's why we see some weirdly whiplash messaging.