[bcjdjsndon]

> there's no cryptographic or architectural way to give someone full weights while withholding the nefarious capabilities those weights encode.

This is true of closed weights, and in fact the problem is worse because they cannot even be scrutinized. We should ban closed weight AI for the very reasons you have just given

[eddyg]

Constitutional classifiers go a long way to reducing unsafe usage in closed-weight models. And like we saw with Fable, closed models can be revoked and classifiers updated when “jailbreaks” are found.

Having the weights gives you the exact affordance an unlearning attack requires, without rate limits.

[bcjdjsndon]

Ps llms have cheated for years, this most recent tripe is an AI vendor trying to hype up its next word predictor in a market of very samey next word predictors, before someone smarter than them eventually figures out how to do this (training + inference) on consumer hardware and kills the market for cloud ai

[bcjdjsndon]

Stick those same classifiers (that you admit dont seem to work) on the open models, and done.

[eddyg]

Classifiers are policy enforced by the process serving the model.

Input classifiers get applied before it reaches the model so somebody hacking an open-weight model would skip this. Streaming classifiers get polled during decoding; hackers delete this check in the sampling loop.

But both are always applied in closed weight models.

Set Llama Guard to 1.0 and nothing is ever unsafe.

[bcjdjsndon]

More than one way to do guard rails, slopboy

[eddyg]

All of which are easily bypassed in open weight models; see my previous comment about K2.5.

[bcjdjsndon]

Using this as your argument is dangerous. You're saying AI is inherently dangerous and an emergency stop button is actually what you need to be safe from the dangers... Forget the fact your open weighted model could be ran from a central location