[chr15m]

It's it just me or does the graph of LLM language performance versus training set size show the opposite of what they are saying? To me it looks flat, implying training set size has little influence on LLM performance in the language. For instance some niche languages appear to out-perform better known languages (with more variance in the niche language performance).

[csomar]

It is not only you, but I think it is only you and me. I've also skimmed through the comments and wondering if they are AI generated; or the people even read the article. The author essentially took a graph, and then claimed a different interpretation to reality.

What the graph shows is that LLMs struggle with "hard" languages (Rust, Go, C#) with the exception of Ruby.

[chr15m]

Good to know, thanks for your reply!