[MadsRC]

Very interesting!

I’ve got an internal tool that we use. It doesn’t do the deterministic classifier, but purely offloads to an LLM. Certain models achieve a 100% coverage with adversarial input which is very cool.

I’m gonna have a look at that deterministic engine of yours, that could potentially speed things up!

[schipperai]

cool - which models are you seeing 100% on adversarial input? I'd love to see the benchmark if you published it somewhere. In my recent sessions while building nah, the deterministic layer handled about 95% of inputs with zero latency/tokens over 13.5k tool calls, 1.5 days of coding, 84% allowed, 12% asked, 5% blocked. All decision logged to ~/.config/nah/nah.log - so you can audit its efficiency