Hacker News new | ask | show | jobs
by dwohnitmok 1168 days ago
The conclusions start from https://github.com/E-xyza/Exonerate/blob/master/bench/report...

This is particularly impressive for Elixir, which is not a language that is a particular focus of GPT-4. I imagine the accuracy for Python is extremely good. Maybe near perfect for this kind of benchmark if allowed to see error messages.

1 comments

It's also possible GPT-4 is better at writing Elixir since there are less beginners/students writing Elixir code and polluting the training data with bad practice or faulty code.
This might not be an issue for the same (somewhat inscrutable) reason that GPT-4 has quasi-perfect grammar.