Hacker News new | ask | show | jobs
by lennoff 1814 days ago
Oh, it does... It generates code that's vulnerable to SQL injections, XSS, etc. It was trained on such code! (hopefully this will improve)
1 comments

Training on code that unintentionally has vulnerabilities is a problem, but I'm even more worried about bad actors intentionally putting code with vulnerabilities on GitHub with the hope that it will become training data. Bad actors might learn how to disguise code to sneak it into Copilot (if disguise is even necessary) and introduce backdoors, etc. It could be especially dangerous because of the "stamp of approval" Copilot has from GitHub/Microsoft. People who would not copy/paste code from the web might feel a false sense of security using Copilot.
I totally expect this will happen. I bet this is already happening. As long as they continue to train Copilot on third-party code that wasn't thoroughly, manually vetted by them, this vector of attack will remain open - and mitigating it falls into the domain of... spam filtering.