Hacker News new | ask | show | jobs
by thot_experiment 19 days ago
I feel like I'm taking crazy pills with hy3, it's either benchmaxxed to hell and back or skill issue on my part but I'd rather use dense gemma. I don't think there's a single model that's wasted more of my time in recent memory.
2 comments
SwellJoe 19 days ago
The Hy3 preview has been a mediocre performer in my benchmarks of security auditing with models, and yes, it is outperformed by Gemma 4 (31b soundly beats it, the MoE does slightly better, even at 4-bit quantization when using the QAT version). Qwen 3.6 27b also beats it.

I'll try it again now that it's out of preview and has been updated with more post-training. It presumably can't be worse, so maybe it's better enough to compete with a 31b model.

link
andai 19 days ago
For context Hy is 295B, roughly on par with DeepSeek V4 Flash (284B).

I haven't tested it yet so I cannot comment on the quality (nor the comparison with 10x smaller (!) models)

link
thot_experiment 19 days ago
There are extreme diminishing returns in real world performance as models get bigger. 10x bigger might mean 5-10% better on benchmarks, a margin that can easily mean it's functionally equivalent in real world use or even a worse performer depending on the context it's being used in, and how good you are at providing meaningful context.

Of course the bigger model embeds more knowledge, but when neither model has the knowledge necessary to perform the task, hy3 makes idiotic decisions all the time whereas gemma 31b has a decent hit rate.

hy3 feels like someone who's read a lot of books and says the right words but has nothing of substance between their ears, gemma feels like a reasonably intelligent person who doesn't understand the domain, the latter is muuuch easier to work with than the former.

link
SwellJoe 19 days ago
Gemma 4 is the first really small model that feels smart, to me. I mean, Qwen 3.6 is arguably better at some coding tasks. But, Gemma 4 has shockingly good reasoning for a small model. Even the tiny 12B, at 7GB on disk in the 4-bit QAT quant, feels like a really big model of a couple years ago. It's a good tool user, can search the web (when given the appropriate skill or MCP), has good vision capabilities, and pretty good prose.

I've only used the Hy3 preview, so I don't want to judge too harshly, yet. But, I wasn't very impressed with it a couple of months ago.

link
nik736 19 days ago
Gemma 4 31B is underrated. It surprises me a lot.
link