[scrollaway]

I guess I thought this should be obvious to everyone but, looking at code and finding exploits is completely different from .. writing exploits.

For one thing exploits often require completely different parts of the code to chain together. Sometimes parts of code the LLM itself isn’t writing.

And, LLMs are ALREADY trained negatively against writing buggy or exploitable code.

[meowface]

It's just an incremental thing. You're both right. They will slowly become less and less likely to introduce vulns due to higher intelligence and better RL. Offensive capabilities will still probably scale faster than automatic defensive-while-coding ones.

[sillyfluke]

>I guess I thought this should be obvious

People in this thread are talking past and misunderstanding each other and making unrelated points.

The point of the response to the top level comment was questioning the conflict of interest in model providers creating separate revenue streams for themselves by selling a product that fixes problems their other product created, akin to OS providers selling anti-virus software back in the day.

Similarly, it should be obvious to you that a software engineer can trivially get into the mindset of writing more expoitable code by pretending the production code they're tasked with writing is hobby code or prototype code.

If profitable revenue streams with adverserial products are in place, no one should be surprised when model providers are disincentivised to improve the "garbage code quality, but hey it works!" nature of their most used code generators.

>And, LLMs are ALREADY trained negatively against writing buggy or exploitable code.

...it should also be obvious people in this forum have wildly different experiences with respect to the code quality the LLMs they use generate. I personally find it difficult to find anyone that argues that the LLMs they are using are consistently generating high-quality code across a vast codebase.