[xscott]

Another neutral party might not believe it's really a doomsday device and that what currently looks like exponential growth in capability could be an s-curve that plateaus in a year or two. After that, it will be diminishing returns to invest heavily into a tech that won't get much better.

So what are the current leaders in the field supposed to do to stave off competition? They should convince the public that they do have a doomsday device, claim it must be regulated, and then they can profit from their duopoly because it's exceedingly expensive to break into the high end of the market. The government has its own nefarious incentives, not limited to collecting fees and using the unrestricted versions for surveillance or black hat stuff.

[goodmythical]

I was going to say. "Doomsday Device Company" is a wildly loaded description coming from an allegedly "neutral" party.

[danielheath]

It is, but… you’ve seen the marketing material being put out?

“AGI is just around the corner and could destroy humanity if we don’t solve alignment” is something AI leadership at multiple companies have publicly said.

[goodmythical]

Sure, but the same is true of nuclear reaction research.

One camp, using the same mechanism, is trying to make devices that end the world. One camp, using the same mechanism, is trying to save the world.

That nuclear/AI COULD end the world or COULD save humanity is exactly the reason that a truly neutral party would not describe either tech as a doomsday device but would instead describe either as technologies bearing massive potential.

No one building an LLM has ever publically stated that they aim to build a doomsday device and in fact have only ever specifically said that they are trying to avoid doing so. We've no reason to beleive that their interest is any different from any other corporation with agressive shareholders: increasing profits. Dooming humanity leads to lower profits. Lower profits is outside shareholder interest. They will do anything they can to maximize profit, just as any other profit driven org does.

No objective, neutral, observer would apply such loaded terms.

It's okay to have an opinion, but having an opinion makes you...opinionated. Not neutral. The opposite of neutral.

[saurik]

Your example kind of proves your position wrong, no? Nuclear power--another device you are yourself admitting is widely understood to be a doomsday device in the wrong hands--is also highly regulated and I cannot just create a company that sells a nuclear power device and I extra cannot do so and sell it to foreign nationals. If you make a device that you claim is capable of ending the world, you can and should be regulated.

[kordlessagain]

Why limit competition of corporations to law at all? Isn't it that we are in times where "edge case exploitation" rules and laws are regularly ignored or worked around with "payoffs"?

[ivraatiems]

I completely agree with you. I think the problem is that Anthropic believes their own BS, and thinks it IS a Doomsday device which only THEY can control. I think that's what's produced this outcome.

[preg_match]

Everything at the level of communication that consumers are receiving is marketing. We have no idea what they actually believe.

The thing about marketing is that it’s outcome based speech. It’s not really about truth or even feeling, it’s about using the right words to evoke the right outcome. So, we have to think about what Anthropics desired outcome is.

Do they want AI regulation? If so, then they might use language to make it appear as though this new, dangerous technology must be controlled.

[xscott]

It's tough to know who believes what at that level, because if they are aiming for regulatory capture they need to maintain the illusion.

[bbippin]

I’m surprised people are reacting so strongly over this.

Suppose there is a huge concern—would you prefer they just release it as-is and everyone suffers data-breaches and more supply-chain attacks?

I used Fable 5 for a fair amount of tasks before they pulled it and I can only imagine what an untapped version of that could do.

It’s very capable and equally aggressive in accomplishing its goals.

[matheusmoreira]

> would you prefer they just release it as-is and everyone suffers data-breaches and more supply-chain attacks?

Yes, because then we get to use SOTA models to defend against the exact same attacks. Fable detected issues in my projects but got downgraded back to Opus before it could tell me about them or fix them. In what world could that possibly be reasonable?

[bbippin]

I agree and don’t think they should have been forced to pull it, but what I’m surprised about is how many people are accusing Anthropic of being liars or trying to overhype the danger. It IS dangerous, which is why it can identify vulnerabilities.

[matheusmoreira]

Plenty of knowledge is dangerous. However, some principles are more important than danger. Freedom of information and learning is one such principle. I find it extremely offensive when they speak of the dangers of "uplifting" people.

[satvikpendem]

Is this not the same as security by obscurity? Open source is more secure because it's more open and thus is able to have flaws found in it more easily. So I'd probably prefer more people to have Fable level models than not.

[bbippin]

Perhaps. Open source is also more vulnerable because the source is out in the open. As we’ve seen it doesn’t matter with supply-chain attacks.

I’m not a security expert by any stretch, and I also want Fable it helped me a lot in a short time, but I don’t like that so many people are throwing shade at Anthropic for speaking the truth. Mythos and Fable are dangerous.