[mtlynch]

>As an example, AutoCodeRover successfully fixed issue #32347 of Django.

This bug was fixed three years ago in a one-line change.[0] Presumably the fix was already in the training data.

[0] https://github.com/django/django/pull/13933

[dboreham]

I wondered that too, but the fix it produces is not the same.

Another thing that seemed odd is the English style used in the responses (watch the video full screen and you can read it).

[lewhoo]

My understanding is all of the issues on SWE-bench have at least a corresponding pull request.

[skywhopper]

The important detail is when the problem was solved. If it was three years ago, then it was likely captured in the training data for the model.