[ivraatiems]

Maybe this will be simpler for Anthropic to understand if they take their own high-minded philosophical nonsense and ego out of it and consider it the way a neutral party would.

Suppose a company calls themselves The Doomsday Device Company. They make and sell excellent-quality doomsday devices. They regularly go online to proclaim that their doomsday devices are the best and most powerful, and also that doomsday devices are dangerous and should be regulated.

The Doomsday Device Company then says they have the world's best doomsday device. (They don't, but they claim they do.)

The US Government hates the Doosmday Device Company for various political reasons, but also has a vested interest in there not being a massive proliferation of doomsday devices.

The Doosmday Device company spends a great deal of time and money telling everyone: "Our doomsday device is the most doomy of all time!" (though it probably isn't) and "Everyone can use it!" (for a lot of money)

It is completely logical, then, for the US Government to say: No, everyone cannot use your doomsday device, because doomsday is bad. (While also meaning: Only we should be able to use it, and you shouldn't be able to tell us how.)

If you do not want to be in the business of having your doomsday devices shut down by the government, well, it would help if you didn't so loudly and aggressively proclaim how doomy they are. It doesn't matter how trustworthy you claim to be, given that your business is making evil doosmday devices. You still won't be trusted!

[xscott]

Another neutral party might not believe it's really a doomsday device and that what currently looks like exponential growth in capability could be an s-curve that plateaus in a year or two. After that, it will be diminishing returns to invest heavily into a tech that won't get much better.

So what are the current leaders in the field supposed to do to stave off competition? They should convince the public that they do have a doomsday device, claim it must be regulated, and then they can profit from their duopoly because it's exceedingly expensive to break into the high end of the market. The government has its own nefarious incentives, not limited to collecting fees and using the unrestricted versions for surveillance or black hat stuff.

[goodmythical]

I was going to say. "Doomsday Device Company" is a wildly loaded description coming from an allegedly "neutral" party.

[danielheath]

It is, but… you’ve seen the marketing material being put out?

“AGI is just around the corner and could destroy humanity if we don’t solve alignment” is something AI leadership at multiple companies have publicly said.

[goodmythical]

Sure, but the same is true of nuclear reaction research.

One camp, using the same mechanism, is trying to make devices that end the world. One camp, using the same mechanism, is trying to save the world.

That nuclear/AI COULD end the world or COULD save humanity is exactly the reason that a truly neutral party would not describe either tech as a doomsday device but would instead describe either as technologies bearing massive potential.

No one building an LLM has ever publically stated that they aim to build a doomsday device and in fact have only ever specifically said that they are trying to avoid doing so. We've no reason to beleive that their interest is any different from any other corporation with agressive shareholders: increasing profits. Dooming humanity leads to lower profits. Lower profits is outside shareholder interest. They will do anything they can to maximize profit, just as any other profit driven org does.

No objective, neutral, observer would apply such loaded terms.

It's okay to have an opinion, but having an opinion makes you...opinionated. Not neutral. The opposite of neutral.

[saurik]

Your example kind of proves your position wrong, no? Nuclear power--another device you are yourself admitting is widely understood to be a doomsday device in the wrong hands--is also highly regulated and I cannot just create a company that sells a nuclear power device and I extra cannot do so and sell it to foreign nationals. If you make a device that you claim is capable of ending the world, you can and should be regulated.

[kordlessagain]

Why limit competition of corporations to law at all? Isn't it that we are in times where "edge case exploitation" rules and laws are regularly ignored or worked around with "payoffs"?

[ivraatiems]

I completely agree with you. I think the problem is that Anthropic believes their own BS, and thinks it IS a Doomsday device which only THEY can control. I think that's what's produced this outcome.

[preg_match]

Everything at the level of communication that consumers are receiving is marketing. We have no idea what they actually believe.

The thing about marketing is that it’s outcome based speech. It’s not really about truth or even feeling, it’s about using the right words to evoke the right outcome. So, we have to think about what Anthropics desired outcome is.

Do they want AI regulation? If so, then they might use language to make it appear as though this new, dangerous technology must be controlled.

[xscott]

It's tough to know who believes what at that level, because if they are aiming for regulatory capture they need to maintain the illusion.

[bbippin]

I’m surprised people are reacting so strongly over this.

Suppose there is a huge concern—would you prefer they just release it as-is and everyone suffers data-breaches and more supply-chain attacks?

I used Fable 5 for a fair amount of tasks before they pulled it and I can only imagine what an untapped version of that could do.

It’s very capable and equally aggressive in accomplishing its goals.

[matheusmoreira]

> would you prefer they just release it as-is and everyone suffers data-breaches and more supply-chain attacks?

Yes, because then we get to use SOTA models to defend against the exact same attacks. Fable detected issues in my projects but got downgraded back to Opus before it could tell me about them or fix them. In what world could that possibly be reasonable?

[bbippin]

I agree and don’t think they should have been forced to pull it, but what I’m surprised about is how many people are accusing Anthropic of being liars or trying to overhype the danger. It IS dangerous, which is why it can identify vulnerabilities.

[matheusmoreira]

Plenty of knowledge is dangerous. However, some principles are more important than danger. Freedom of information and learning is one such principle. I find it extremely offensive when they speak of the dangers of "uplifting" people.

[satvikpendem]

Is this not the same as security by obscurity? Open source is more secure because it's more open and thus is able to have flaws found in it more easily. So I'd probably prefer more people to have Fable level models than not.

[bbippin]

Perhaps. Open source is also more vulnerable because the source is out in the open. As we’ve seen it doesn’t matter with supply-chain attacks.

I’m not a security expert by any stretch, and I also want Fable it helped me a lot in a short time, but I don’t like that so many people are throwing shade at Anthropic for speaking the truth. Mythos and Fable are dangerous.

[threatofrain]

It's quite a bit more complicated than that. The popular narrative is not exactly on Anthropic, as the general public is far more aware of OpenAI than Anthropic. The narrative is on AI and whether everything we know about society is going to change.

Also, as far as priorities and worries go, for most people cybersecurity is way down the list.

[gerdesj]

"It's quite a bit more complicated than that."

Ohhh no it isn't! (Ohh yes it is) etc

"as the general public is far more aware of OpenAI than Anthropic"

I run LLMs on my own gear with llama.cpp (compiled from source) and I could not tell you anything about either company except they fiddle with AI stuff and that (I don't actually care). I glaze over on news about both organisations in equal measure on mention.

I think you'll find that the general public would not be able to name either company without being asked to pronounce their name from it being written down.

[sillysaurusx]

My eighty year old father brought up OpenAI unprompted a few months ago. At this point it’s hard to find anyone who hasn’t heard of OpenAI.

[jazzyjackson]

Depends entirely on whether they leave CNBC playing as background noise

[DANmode]

Yes. Can’t wag the dog without centralized networks; even if they’re now Meta algorithms, they’re still there.

[gerdesj]

I think my point still stands 8)

I know plenty of people who will put the letters A and I together and get rather confused about what on earth is going on but very few of those would mention any co apart from Microsoft, Google, Facebook or errm Twitter.

No one has a bloody clue about all this stuff apart from us lot and we have no real idea about it either!

Oooh tulips!

[threatofrain]

Is your point that the name people actually remember is ChatGPT? Okay, let's say the actually popular name that threatens to become the Kleenex of LLMs. ChatGPT.

[satvikpendem]

I'm not sure what your personal anecdote has to do with general trends, there have always been hermits in history. I also run local models with llama.cpp but I actually stay abreast of the issues in the field as we may similarly be impacted by recent actions.

[gerdesj]

Silly me and my silly stories!

HN is a forum and as far as I am aware, anecdata is allowed here. You might disagree with me and my experience and that is fine too but please don't denigrate me.

[satvikpendem]

It is a forum, yes, and anecdotes are fine, as I said I do the same as you wrt local models, but that doesn't mean an irrelevant non sequitur is useful to other readers, and pointing it out is not equivalent to denigration.

[ajmurmann]

Well, regulating doomsday devices is a reasonable thing to want. A reasonable regulation of such devices would call for proper safeguards and safety testing. I think Anthropic would have been fine with that.

Instead what happened is a one-off nationalist decree that solves none of the two concerns.

[estearum]

Yes this is also why if you're building a startup in the nuclear power space (empirically demonstrated Doomsday Device), then you can expect USG to come in and apply arbitrary, opaque, and unexplained rules on you, steal your assets, and destroy your business. And also probably not do that to any of your competitors who are doing the exact same thing as you.

[s3p]

Shame on a company for sticking to their values, I guess.

The dichotomy between Anthropic and OpenAI's treatment honestly couldn't be more obvious. OpenAI has also asked for increased AI regulation, and they've also released GPT 5.5 Cyber which is claimed to have the same vulnerability-finding abilities as Mythos. OpenAI received no such notices like Anthropic. OpenAI also received a government contract, while Anthropic was banned from DoD use.

Regardless of your thoughts about Dario or his company, this treatment is obviously not based in any rational principle, and pretending it is would be stupid.

It's only a matter of months before the open source models achieve this same capability. What is the US government going to do then? Ban all people in the world from accessing the Chinese models? If you think about these arguments for more than five minutes they really do fall flat.

[matheusmoreira]

Their values are garbage. Paternalism, censorship, suppression. These are people who think they are the enlightened ones while we're all dangerous terrorists. They are in fact pulling up the ladder behind them, just like all the other big techs.

It's indeed quite satisfying to watch them be the first ones burned by the heavy hand of government they worshipped so much.

[andai]

Okay, but hasn't OpenAI been doing the same thing for years? They seem to be on slightly better terms though...

[kordlessagain]

Consider Dario was VP of AI research and oversaw alignment and safety at OpenAI. He left in 2020. In my opinion and from direct observation, it takes time to change policy at large well funded companies, so even after he left, his influence was still being felt.

[amazingamazing]

Yes, albeit not to the dame extent

[SilverElfin]

Well when OpenAI did it most prominently, it was actually Dario Amodei who did it while working at OpenAI. Although you are correct that OpenAI has also pushed for safety regulations. I doubt Sam Altman is part of the effective altruism cult though - he’s more obviously looking for regulatory capture to give him moats.

The recent OpenAI post calls for mandatory safety certification of all frontier level models:

https://openai.com/index/frontier-safety-blueprint/

[lazide]

OpenAI is happy to sell their device to the gov’t to blow things up with, Anthropic tried to tell the gov’t to pound sand, no blowing things up for you.

[slopinthebag]

It was more like "you can use our device to blow things up you just need a human to type --dangerously-skip-permissions when they run it"

[SilverElfin]

To be clear, Anthropic was completely okay with their tech being used in war, including current conflicts. They just wanted a human making some decisions. The military didn’t want a vendor giving them restrictions on how to operate a tool, and since Anthropic’s restrictions could be a problem anywhere they appear among other military vendors as well, the government used the supply chain risk designation to say “this can’t be anywhere in our tools”.

[ajmurmann]

After a contract had been negotiated with the exact terms that Anthropic pushed to keep in place.

[SilverElfin]

We don't have the actual contracts publicly. So I bet it has terms that let the government do what it wants ultimately by naming exceptions. That way OpenAI can claim it has some controls in place while the military has the freedom it needs practically.

[ajmurmann]

I'm referring to the contract between Anthropic and the DoD from July 2025 in which both parties agreed to Anthropic's acceptable user policies which then later were seen as unacceptable despite the same admin having signed them.

[satvikpendem]

David Sacks corroborates this: https://twitter.com/DavidSacks/status/2065853007619588171

[adahn]

When I read this comment I don't get "Neutral Party" from it. I'm finding it hard to decide whether to focus on the loaded characterisations or (my best guess at) the underlying substance.

To address the point I think you (and the article) are making; there's a difference between advocating that a scoped, due-process-protected power exist and endorsing any given exercise of it. This point is made in Anthropic's original statement, but it's seemingly been missed by everyone taking a position against them.

>As we have stated publicly, we believe the government should have the ability to block unsafe deployments, as part of a statutory process that is transparent, fair, clear, and grounded in technical facts. This action does not adhere to those principles. [1]

But with respect to your specific post:

>high-minded philosophical nonsense

Taking a sophisticated approach to a problem will generally improve the outcome. I suppose your point is that it can be difficult to deliniate sophisticated & good-faith reasoning from self-serving rationalisation. Much of what I've seen from Anthropic supports the former. Perhaps others could do more to build up the case for the latter.

You can look at things like the circumstances that the company was founded in, the lack of political cozying with the current admin, the lack of controversies or disgruntled ex-employees, them taking a stand against the DoD (against their business interests), positions they've taken on various issues including data centres in gulf states.

>Suppose a company calls themselves The Doomsday Device Company

Anthropic hasn't. What's the explanatory purpose of adding this to the analogy?

>They make and sell excellent-quality doomsday devices | They regularly go online to proclaim that their doomsday devices are the best and most powerful, and also that doomsday devices are dangerous and should be regulated.

Anthropic invests a reasonable amount into assuring the safety of the products they sell. No-one is claiming they sell doomsday devices. Concerns of doom relate to future iterations of the underlying technology.

>The Doosmday Device company spends a great deal of time and money telling everyone: "Our doomsday device is the most doomy of all time!"

They have what's essentially an automated vulnerability-discovering machine. They've done their best to be open about the very real implications of general availability of a system like this. Using the word "doomy" positions them as childish and unsophisticated for doing this, when it's actually the reasonable and responsible thing to do.

1: https://www.anthropic.com/news/fable-mythos-access

[SecretDreams]

How does this look if it's a perpetual motion machine instead of a doomsday device?

[knollimar]

>(They don't, but they claim they do.)

I hate to shill them, but wasn't mythos/Fable SOTA?

Your main point still stands without this aside

[cassianoleal]

What does this question mean? Of course they are state-of-the-art. After all, they are the most recent and most advanced models out of Anthropic. If/when they release a new version of their models, Mythos/Fable will cease to be state-of-the-art, as the new ones become it.

[nearbuy]

It means Mythos/Fable was the strongest model globally, not just the newest model from a company.

[knollimar]

Yeah if the analogy is LLMs are doomsday devices it's hard to say mythos wasn't the best. That was my point

[Barrin92]

>It is completely logical, then, for the US Government to say: No

Not sure about that one given that the US government just reversed a ban on the exports on the very chips to China that enable said technologies, you don't hear so much about the chip wars any more.

I think entrepreneurs largely approached this administration with the attitude that if you're running Doomsday Incorporated they aren't going to say, "no, don't export that", but "hell yeah baby, how do we get a 25% cut on every sale?" because that's quite literally what they did on the hardware front.

I mean I have no strong opinion on whether Antrophics statement are true or smart, but the idea that it was regulated because someone in this administration thought it posed incalculable risks is a bit funny, i"m pretty sure they wear that as a logo printed on their t-shirts, it seems to be the sole guiding principle of their foreign policies. Palantir has successfully used doom-mongering as an advertisement strategy for a decade or so

[ivraatiems]

In fairness, I think this might be a case where the pure id represented by this administration happens to align with the correct and logical choice. You could be right, but for this instance, the outcome is the same.