[jlarocco]

I get that, but I meant how often does it introduce bugs in the situation the OP mentioned, using it for "a language or framework you don't usually use", where I won't know if it looks right or not because I'm unfamiliar. Are they going back to fix the code often?

Personally, I don't even rely on normal autocomplete very much when I code because I feel like if I don't know what I'm trying to call, then I don't know the language or library well enough, and it's best to go look at the documentation.

[lolinder]

I recently used Copilot while learning Rust and it was immensely helpful for the learning process. There were a lot of language features that I learned about by seeing Copilot use them.

As for mistakes, I've got a good enough sense for reading code in any language to know if something is obviously wrong with an algorithm. The mistakes Copilot makes aren't usually to do with language-specific syntax, they're typically algorithmic and therefore easy to spot in any language. This makes sense if you think about GPT-3's output for English: it tends to be syntactically correct, but often completely wrong about facts.

It helps that Rust's compiler is so picky and so helpful, because on the rare occasions when Copilot was completely wrong about the language the compiler could set me straight pretty quickly. Writing automated tests also helped and was made easy by Copilot. I could write a single test case showing how to use the API, and then just write function names to generate the rest of the tests (obviously I would sanity-check the assertions Copilot produced).

[baby]

I'm not sure, I use it A LOT for cryptographic code in Rust and I don't blindly rely on its output. I believe if there's a bug that was introduced it would be my fault with high probability, not Copilot's.