[abotsis]

China releasing open weight models, powerful or not, does nothing to prevent their development of models they’ll use for evil. Nor does it stop someone from abliterating a non-Chinese model and using it for evil.

Sorry- I don’t see any other reason aside from Anthropic protecting their own interests.

[ianm218]

What are you arguing against? He doesn’t advocate for a ban on Chinese open source or non Chinese open source.

[abotsis]

I’m simply making the point that any model can be abliterated to sidestep guardrails. It’s similar to the argument that open source software like nmap or netcat shouldn’t be allowed because it can be used for evil.

To continue the analogy, he’s basically asking for some magic guardrails in nmap that doesn’t allow it to be used for scanning a network you don’t own. Of course even if you could add that- you could modify the source to bypass it.

It’s basically “open weight models are a public good if you can guarantee safety” .. which you can’t. So the only viable alternative is a hosted nmap that requires you to prove you own a network before scanning it. Which is impossible. So it’s all the right words and sounds nice, but making an impossible ask.

[ianm218]

Meh you gotta go for progress over perfection. Screening model releases is progress, you know that companies aren't mass distributing dangerous releases. There might be some foundational security work in terms of how to make models hard to modify after that is possible.

When the risks are as high as discussed here it doesn't really make sense to give up because you haven't found the silver bullet.

[culi]

Is it possible to abliterate a closed model? I was under the assumption that you needed the weights

[abotsis]

No- and that’s the point. Any open weights model, regardless of the origin can be abliterated. So effectively he’s saying he’s all for open weights models as long as they’re safe, which by the very nature you can’t guarantee.