I think copilot is the wrong application of AI. It spits out what most coders would write for a specific problem. First, if many people have the same problem, than libraries are the solution, not copy-pasting. Also, just because many people do one thing doesn't mean it is the right thing to do, and you sometimes get code with security vulnerabilities.
Instead, I would like a system telling me about obscure things, traps, vulnerabilities, performance issues, etc... like the machine learning linter. The way I could see it work is by matching my code with bugfix commits. For example if several commits replaces "printf(buffer)" with "printf("%s", buffer)" and I write "printf(buffer)", I want an AI to tell me "code like yours is often replaced in commits, it may be wrong", bonus points if it can extract the reason from commit messages ("format string vulnerability") and suggest a replacement ("printf("%s", buffer)"), mega-bonus if it can point me to good explanation of the problem.
Pissing lines of code is easy, I can do it, anyone with a couple weeks of training can do it, I don't need a bot to help me with that. Thinking about everything while I am pissing my lines is hard, and I will welcome a little help.
A nice thing about that approach is that it is unlikely to result in worse code than what I would have written by myself, because it will be designed to trigger only on bad code.
I'm sure there's an IDE out there which will do that already without any AI. Just need to lint your code, highlight the bad stuff it finds and suggest a refactoring.
Most of them already do, personally, I use SublimeLinter for SublimeText, and LSP support.
But linters work with hand crafted static rules, which is good and the idea is not to replace them. The idea is to used big data techniques to find unwritten rules based on commit histories, the idea being that we are more likely to remove bad code than good code. So if your code looks like code that is often removed, is is most likely bad, even if it doesn't match an explicitely written anti-pattern.
Sounds good, although it would have to be context aware. For example, code that often gets removed in a production environment might be dissimilar to choose that is typically removed in dev or testing.
There are also other triggers of code removal and refactoring that are outside the code base, such as an organisation migrating to a different platform. An AI trained on a large public commit history could encourage a general shift towards already-established big players, punishing smaller organisations.
I agree with your objective, however it's obvious why Microsoft didn't do this: they wouldn't have been able to make good on their billion-dollar investment in OpenAI/GPT-3, which they REALLY want to justify.
I don't think Copilot is useless at all. Today it's actually been very helpful for me with interactive, notebooks-based programming. And it's also just an early beta right now; as the model improves and the tooling around it matures a little more, I suspect I'll be using it a lot for interactive stuff.
Notebooks programming has a flow of "execute a small bit of code, check the results, and iterate", and this fits perfectly with Copilot since you still need to check if the suggestions work.
Maybe this kind of programming is where Copilot finds a niche, maybe not. I don't know. I'm skeptical of its use in larger applications where you can't trivially check if the code you wrote (with its help) did what you want. I think there needs to be a lot more tooling built around that to really make it compelling for larger applications like that, likely in the form of more editor tooling integrations. But I think it's promising. I wrote about that a little more here: https://phillipcarter.dev/posts/four-dev-tools-areas-future/...
Instead, I would like a system telling me about obscure things, traps, vulnerabilities, performance issues, etc... like the machine learning linter. The way I could see it work is by matching my code with bugfix commits. For example if several commits replaces "printf(buffer)" with "printf("%s", buffer)" and I write "printf(buffer)", I want an AI to tell me "code like yours is often replaced in commits, it may be wrong", bonus points if it can extract the reason from commit messages ("format string vulnerability") and suggest a replacement ("printf("%s", buffer)"), mega-bonus if it can point me to good explanation of the problem.
Pissing lines of code is easy, I can do it, anyone with a couple weeks of training can do it, I don't need a bot to help me with that. Thinking about everything while I am pissing my lines is hard, and I will welcome a little help.
A nice thing about that approach is that it is unlikely to result in worse code than what I would have written by myself, because it will be designed to trigger only on bad code.