[corpdrone2021]

I feel like a mix of hand written test cases and copilot generated code might go somewhere, but I think you've got the basic problem sorted out. I'd much rather type an algorithm in from scratch than wrap my head around whatever copilot spits out.

[dkersten]

I had an idea long ago that you basically write unit tests (nowadays I would add property-based tests to the mix too) and a genetic algorithm (best I could come up with at the time, nowadays we obviously have much fancier techniques, as evidenced by Copilot) would come up with code to try and make the tests pass.

I could see Copilot used in such a way. I think the interaction would have to change though: force the user to give it the tests as input, not give it some basic instruction, have it generate code, and then I try to write tests after. The tests should be the spec that Copilot uses to generate its output.

Right now, I'm not excited about Copilot. Like you say, understanding what Copilot spits out is difficult and I suspect more error prone than just writing it yourself (since we often see what we want to see and can overlook even glaring mistakes). I'm also not excited about them ignoring the licenses of the code they trained on. But I can imagine a future iteration that generated code to pass some tests that I could get excited about.

[corpdrone2021]

It seems to me that "generate the code that makes these unit tests pass" is actually a much saner engineering task than "go from a comments to an implmentation"