Hacker News new | ask | show | jobs
by film42 310 days ago
Is there a crowd-sourced sentiment score for models? I know all these scores are juiced like crazy. I stopped taking them at face value months ago. What I want to know is if other folks out there actually use them or if they are unreliable.
3 comments
hnfong 310 days ago
Besides the LM Arena Leaderboard mentioned by a sibling comment, if go to the r/LocalLlama/ subreddit, you can very unscientifically get a rough sentiment of the performance of the models by reading the comments (and maybe even check the upvotes). I think the crowd's knee-jerk reaction is unreliable though, but that's what you asked for.
link
NitpickLawyer 310 days ago
Not anymore tho. It used to be the place to vibe-check a model ~1 year ago, but lately it's filled with toxic my team vs. your team, memes about CEOs (wtf) and general poor takes on a lot of things.

For a while it was china vs. world, but lately it's even more divided, with heavy camping on specific models. You can still get some signal, but you have to either ban a lot of accounts, or read new during different tzs so you can get some of that "i'm just here for the tech stack" vibe from posters.

link
littlestymaar 310 days ago
Yeah, some people just can't stop acting as if tech companies were sport teams, and it gets annoying fast.
link
parineum 309 days ago
I don't really go there much anymore but, when I was, there seemed to be an innordinate amount of Chinese nationalism from young accounts speaking odd English.
link
nurettin 310 days ago
This has been around for a while https://lmarena.ai/leaderboard/text/coding
link
klohto 310 days ago
openrouter usage stats
link
esafak 310 days ago
https://openrouter.ai/rankings

The new qwen3 model is not out yet.

link
setsewerd 310 days ago
Since the ranking is based on token usage, wouldn't this ranking be skewed by the fact that small models' APIs are often used for consumer products, especially free ones? Meanwhile reasoning models skew it in the opposite direction, but to what extent I don't know.

It's an interesting proxy, but idk how reliable it'd be.

link
matznerd 310 days ago
Also, these small models are meant to be run local so not going to appear on openrouter...
link