[micimize]

Any time I see humaneval comparisons I feel the need to point out that humaneval is only 164 questions. 66.5 vs. 65.9% is a difference between 109 and 108 solutions, a single question. Still interesting work though

[moyix]

At least they used HumanEval+, which adds a bunch more test cases and fixes some errors in the original benchmark!