Hacker News new | ask | show | jobs
by Aurornis 17 days ago
> yes. https://www.alphaxiv.org/abs/2607.bonsai-27b table 14 shows that bonsai retains roughly 95% of the fp16 27b model's average performance

I know that's what the paper says the benchmarks say, but these models feel significantly worse than the base model when you start using them for real tasks.

Even the Q4 quant which they put in between their Bonsai models and the FP16 in the benchmarks has a tendency to go into doom loops and get lost compared to even Q5 or Q6.

I don't know how much of this is due to benchmaxxing (putting the benchmarks into the post-training loop) or cherry picking benchmarks to look good. If you spend a lot of time using local models you learn to take vendor provided benchmarks with a huge heap of doubt. Everything looks amazing in the benchmarks these days.