[rowanajmarshall]

> OpenAI estimates that Codex (what powers Github’s Copilot) can already complete 37% of coding tasks.

This makes me a bit suspicious of the article.

Obviously this is just one developers experience now. I've been using Copilot for the last month, and it's an awesome bit of tech. But generally speaking, when I let it write a blob of code:

* There's a 50/50 chance it'll even compile * It's always, _always_ wrong in some way, usually in ways tricky to spot * And the tests it wrote _for it's own code_ are generally wrong too

As always, I think the bottleneck is in describing exactly what you want in a way an AI can successfully interpret it.

[machiaweliczny]

It’s not expensive to check for syntax correctness, so it’s matter of time till they introduce it.