[eskibars]

I've been building a product (https://zeroquarry.com) that can use a variety of models for finding vulnerabilities. One of the things I've noticed is that the models will nearly always comply with some of this, but how you prompt it matters a ton. I've worked on a set of prompts and approaches which rarely get flagged

[gcatalfamo]

Sharing them would be interesting. However, it is getting nonsensical that this is needed.

[eskibars]

What we've actually seen is a couple things that make this impractical "to just share a prompt". First, that nearly every major model still hallucinates a lot of vulnerabilities. Especially with temperature=0.7 as states in the original blog here, you get very inconsistent results regardless of the prompt, but that's almost kind of moot to the bigger picture. What you really need is to override the planning phase beyond asking a model "find the vulnerabilities" and you need to add another 1+ checking phases for "validate these vulnerabilities." Without that, even with the absolute best models with the highest levels of thinking enabled, you end up with garbage.

Setting the prompts and the flow with a coordinator agent directly gives a system much better capability to investigate security issues because it doesn't rely on 1-shotting things