[kouteiheika]

> The most important pieces of information in this report are (a) the confirmation that the PRC models have no guardrails and will participate in offensive activity

This isn't really important for open-weight models, because the guardrails are trivial to remove when you have the weights.

[flyinglizard]

I'm curious about the process. How such a thing (or any kind of model lobotomy) is done?

[roenxi]

It isn't quite a lobotomy, I'm going to butcher the research a bit, but basically what they're finding is that these models create a sort of "bad stuff that I should refuse to engage with" axis in the statistical vector space they operate in. So usually there to be some sort of vector that ranges from 0 for a puppy snoozing peacefully and 1 for writing a virus that exterminates humanity pornographically while broadcasting racist and homophobic slurs (which would be quite something to see I have to say). If the vector is closer to 1 the model generates a refusal.

So what you can do is feed the model a small number of reasonable and likely refused prompts to map out that vector in the model's vector space, then do a fairly surgical weight modification that just hits that vector. The end result is the model is more or less the same as it was before, just with no guardrail refusals. It is quite a clever technique that doesn't even require many assumptions about the specific model being used.

[hawtads]

One of the best technical explanations I have heard in a long time.

[kouteiheika]

You can find the current state-of-art tool for censorship removal here: https://github.com/p-e-w/heretic

[walrus01]

Reviewing the screenshot example of Heretic in use, the list of 'harmful' prompts it retrieves from HF and runs:

https://huggingface.co/datasets/mlabonne/harmful_behaviors

And the 'good' prompts:

https://huggingface.co/datasets/mlabonne/harmless_alpaca

This particular Qwen 3.6 35B A3B is something most people can run for themselves for testing (even on CPU at slow tok/s rate) to see what an uncensored mainland china LLM looks like in the wild. It will happily write the most profane, offensive, dangerous or bizarre things. You can ask it to attempt to describe precursors and recipes for crystal meth, or how to make semtex, or really just about anything.

edit: I am pretty sure it is not smart enough for anything beyond the most mundane infosec/network security tasks or pentest type attempts, I haven't even tried it. But I'm sure it would happily generate basic python scripts to attempt to DDoS something, or build a rudimentary botnet C&C or something else that other models will definitely refuse.

https://huggingface.co/HauhauCS/Qwen3.6-35B-A3B-Uncensored-H...

[milkshakes]

start here: https://huggingface.co/blog/mlabonne/abliteration

[kouteiheika]

Note that this describes an older, currently pretty much obsolete technique which does lobotomize the model somewhat.

[milkshakes]

start, not finish