Independent testing of prismml suggest quite a capability drop off outside of their cherry picked benchmarks. I'll be curious to see what this model achieves though.
"Neutrino-1 8B was trained natively in its shipping format. There is no full-precision product model that was rounded afterward: the ternary representation is the medium the weights learned in, and the training methods that hold this quality at this depth are the lab’s unpublished work. The findings below are the part that travels."
This statement seems misleading at best.
Both the model page and the release page are basically unintelligible - I don't have a ton of faith in the work here, at least PrismML write coherent releases for their models.
Edit: Another beautiful piece of prose here, I almost wonder if they used the 8b model to generate the content for this release...
"Across the 6.95B coded weights, 62.63% sit at zero and the remainder splits 18.68% plus to 18.69% minus: sign-balanced to a hundredth of a point with no constraint asking for it."
> sign-balanced to a hundredth of a point with no constraint asking for it."
This isn't their model, this is (probably?) ChatGPT doing a brag / promo deck authorial voice. It routinely uses half a dozen sentence constructions that are relatively uncommon in normal or technical speech. Eccentric. Persuasive. Trying too hard. Restating its point in a promotional way that doesn't sound natural, leading into a sentence where it hyperbolically sells you on having done the impossible.
Normal persuasive speech uses these constructions, especially public speaking doing a VC pitch or an Ancient Aliens, but it would be fucking strange if a person started chaining them in normal conversation and using little else, trying to insinuate competence.
You start to recognize it pretty quickly on Youtube.
I really had high hopes for the larger Ternary Bonsai and it feels like there is scope to improve, but I get the sense (albeit a naïve, probably not fully informed sense) that improvement can perhaps only come by training directly into ternary.
I’ve actually been really impressed with the 27b model they recently released - amazing performance approaching 40 tok/s on m4 max and I didn’t run into any quality issues in the small set of tasks I tried. Haven’t gone full coding with it yet but suspect it’s better than say a 9b or 12b model.
I ran into some issues that are more extreme versions of Qwen's thinking loops while reasoning. It is great at the small puzzles I set for it but it did some frankly insane things on a PHP coding task I set it.
It also had some issues that might be parsing/chat template stuff, tool calling oddities. I will try it again, I did try it pretty much the day it shipped and it's possible there are more improvements in their llama.cpp fork since.
It would be churlish to be overcritical, mind you — the PrismML ternary stuff is an advance — but it feels like it should be applied at training. I figure we will see that, somewhere, quite soon.
Did you try the BottleCap ThinkingCap Qwen post-train with the reduced thinking overhead?
Not tested much but it is not noticeably worse than the underlying Qwen 3.6 27B in Q4_K_M, which in my experience is kind of a first for a Qwen fine-tune of this nature. They are almost always worse.
I think it does use fewer tokens while reasoning, which is potentially useful. I need to do more testing, because any performance advantage over the 27B is useful for me on an M1 Max.
Sorry I should have clarified - I meant that a ternary 27b model would outperform a non-quantized or 8 bit quantized 9 or 12b model - which it is generally close to (or much smaller than) in size. So yeah the comparison I was trying to make was between models of equivalent size or models that could run on similarly sized hardware.
Ah yeah, that makes a ton more sense :) I mean, what you said earlier also makes sense but was too obvious, now it makes sufficient sense, thanks for explaining :)
"Neutrino-1 8B was trained natively in its shipping format. There is no full-precision product model that was rounded afterward: the ternary representation is the medium the weights learned in, and the training methods that hold this quality at this depth are the lab’s unpublished work. The findings below are the part that travels."
This statement seems misleading at best.
Both the model page and the release page are basically unintelligible - I don't have a ton of faith in the work here, at least PrismML write coherent releases for their models.
Edit: Another beautiful piece of prose here, I almost wonder if they used the 8b model to generate the content for this release...
"Across the 6.95B coded weights, 62.63% sit at zero and the remainder splits 18.68% plus to 18.69% minus: sign-balanced to a hundredth of a point with no constraint asking for it."