[solenoid0937]

Open source AI fails first contact with sufficiently-intelligent-as-to-be-dangerous AI.

The day Mythos class models are open sourced will not be a good day. I don't think you understand the impact that will have on the world and on cyber defenders everywhere. It will be pure chaos.

Even if you don't think Mythos-class is the bar, open source has to stop at some point, you don't hand everyone a superweapon.

[bloppe]

Every single one of those sentences is highly dubious. Cyber-defenders would be pretty jazzed about having easier access to Mythos-class models. Cyber-defense is easier with better tools.

[solenoid0937]

Cyber defenders already have access to Mythos class models at most companies with the biggest user facing products and services. They are using them extensively (including on OSS deps)

Handing every skiddie and nation state and APT and hacker group access to Mythos does not help cyber defenders

Even if you don't think Mythos is a big deal: At a certain point models become smart enough as to be dangerous, and you don't give everyone a superweapon. Open source has an end of the line sooner or later.

[bloppe]

Let's assume for the sake of argument that Anthropic has a monopoly over significantly superior models. They will never be a perfect gatekeeper. The number of black hat hackers with access they shouldn't have will be highly correlated with the total number with access, so to be "conservative" Anthropic would deny access to many white hat hackers who should have access (like my company, which runs cloud infra for hundreds of other companies but is nonetheless too small to be considered). It only takes a few bad apples to ruin the bunch, and once a black hatter has access, all the white hatters without would be exposed. At that point, it would be better for everyone to have access.

Luckily, Anthropic has no such monopoly, so the point is sorta moot

[Der_Einzige]

This line of thinking is ontologically evil, life denying, and it and its believers should be rejected and fought with extreme prejudice.

[solenoid0937]

I'm sorry, you're right! What was I even thinking? Of course we should hand every man, woman, and child a mini nuke!

[layla5alive]

A Mythos class model is not a nuke.

I'd wager 99.99% of humanity would not want to start a nuclear war by vaporizing millions of people, animals, etc. with a nuke. They'd back away slowly and figure out how to ensure it was safely disposed of so they didn't need to be near it or worry about it.

A very small number of people would like to have a nuke with which to harm other people.

Some of those people even probably got that way because of actions taken by people saying things like "if we don't do it, somebody worse will" and "we are the only ones who can be trusted to control this power"

[alightsoul]

Oh remember when the same was said about GPT-2? It will actually force cyber security to be taken seriously instead of just bureaucracy

[solenoid0937]

The main risks called out with GPT 2 were misinformation and they were right about all of it. Every risk they called out came to pass.