Hacker News new | ask | show | jobs
by tarruda 19 days ago
Recently tried the pelican test on GPT-OSS which was probably one of the best local models of 2025. So cool to see how models have improved in the SVG pelican!
2 comments

I'm skeptical, as tests go, I think that's burned out now. They could easily be training specifically to get a better pelican...
Doesn't have to be specific training, just has to consume simonw's blog. It's got lots of SVG pelicans, with helpful commentary on how good they are. I think there might be some kind of hill climbing going on here.
If they were training specifically for it the result would be much better
GPT-OSS was by far the worst local model in 2026. I have the luxury of plenty of H100's to try them, and Gpt-oss was atrocious with a coding agent. Gemma4 and Deepseek 4 are the best for coding, Qwen for images.