[yuntong]

The previous comment was to describe the experiment settings. AutoCodeRover currently generates patches. Auto-generating high quality tests can be a parallel effort and another direction to explore. These efforts can eventually be used together.

[dboreham]

The point is that a patch without a test is not generally a useful thing. How do we know the AI generated patches are valid?

[abhikrc]

We know that by testing the patches. Automated Program Repair is an area of research which has done this.

The patches are tested against a test-suite. So, if there are tests, we welcome them, we definitely use them to validate the patches.

[nfm]

I agree in principle, but if it also generated a test, how would you know that was valid?

The value I get from copilot is the ability to code faster, not the ability to code.