He has also been telling bald-faced lies about open source/open weights models that are easily disproved. For example, he claimed that they lack the collaborative benefits of open source because "we can't see inside the model".
Open weights models are responsible for enabling reams of research on interpretability methods that do just that. And they have facilitated so much collaboration on architecture, inference optimizations, training and steering methods, and other topics that were completely out of reach with closed models like Anthropic's. It's really staggering to me.
The premise (that "powerful models" implicate "safety" concerns that must be controlled by "companies") does not seem true or self-evident to me. This tired fearmongering campaign centered on GPT-4 three years ago, a model now surpassed by open models that run on laptops.
Did fearmongers like Amodei say, "Oops, we were wrong! It wasn't that dangerous after all"? No. Of course they didn't.
1) the company has device-level control to the degree that they can not only restrict which API endpoints people can connect to but which accounts they use to do so (in which case this already isn't an issue); or
2) they don't, and all bets are off anyway, open weights or not.
You mean to tell me that anyone can own a nail-gun? We can't have people buying their own nail-guns, next thing you know they might build things that aren't up to code!
> "Once the weights of a model are public, they cannot be retrieved. If a model possesses dangerous capabilities, it is permanently out in the wild... We need to consider regulatory frameworks that account for the unique risks of open-source distribution of highly capable frontier models."
Curious if I'm being downvoted by people who think there will never be a model with dangerous capabilities? Or people who think when it happens we shouldn't add guard rails?