[btilly]

How does this make the model useless? It finds and fixes the security hole. It can even write a test that verifies that the fix didn't break things. But it deliberately doesn't reveal the fact that it was a security issue that was fixed.

Seems useful to me. But more useful for defenders than attackers.

[7734128]

Imagine that you have the repo A, ask the model to "fix the security issue" and end up with A'.

Just take the Diff A' - A to see the security hole.