Hacker News new | ask | show | jobs
by falcor84 587 days ago
I feel a bit bad bringing this up, but should Gemini actually be considered SOTA?

They make impressive demos, but I can't recall any of their released models being at the top of any leaderboard.

EDIT: Sorry, looking into it a bit more now, they still seem to be at the top in term of the context window, so they got that going for them.

1 comments

Leaderboards are misleading. Try diff models for YOUR task and you’ll see a wide variety of outputs compared to “official” rankings.
Ok, maybe I haven't experimented enough; so for which tasks is Gemini the SOTA?