[hudsongr]

We assume LLMs can always be fooled, so the judge is just one step of the verification process. The deterministic layers, per-tool db roles, read-only-by-default integrations, query timeouts, are the ones an admin can configure. The judge's rules aren't exposed to admins today. I like your idea of letting admins add their own rules or knowledge to the judge, we should add that.

[Arshad-Talpur]

One of the core issue here is LLMs are probabilistic not deterministic, even with set of rules or context , llms are inherently built to make best guess, so my question is how you are ensuring that LLMs are following rules?

[marinoseliades]

You're right, and we don't. The judge is best-effort screening not enforcement. Enforcement is deterministic with per-tool Postgres roles, proxy-injected creds, host allow-list, human approval on destructive writes. Those hold whether the judge is right or wrong.