[Terretta]

What did you mean by "opposite"?

You seem to be suggesting it got a bit worse, and the aider article seems to suggest gpt4 got a bit worse, although much faster at being a bit worse, while gpt3.5 got worse, then better, while faster.

[reitzensteinm]

The Aider article has been updated with the complete results. Previously Turbo was leading slightly. So far any difference is in the noise.

However, in my opinion the first attempt score is more important, and Turbo does genuinely seem to lead there. There's still a possibility the updated training data has tainted the results.