Hacker News new | ask | show | jobs
by BoorishBears 605 days ago
The novelty of the inexplicable bitnet obsession has worn off I think.
2 comments

IDK, they remind me of Sigma-Delta ADCs [0], which are single bit ADCs but used in high resolution scenarios.

I believe we'll get to hear more interesting things about Bitnet in the future.

[0] https://en.wikipedia.org/wiki/Delta-sigma_modulation

We have yet to see a large model trained using it, haven't we?
Bitnet models are just another piece in the ocean of techniques where there may possibly be alpha at large parameter counts... but no one will know until a massive investment is made, and that investment hasn't happened because the people with resources have much surer things to invest in.

There's this insufferable crowd of people who just keep going on and on about it like it's some magic bullet that will let them run 405B on their home PC but if it was so simple it's not like the 5 or so companies in the world putting out frontier models need little Timmy 3090 to tell them about the technique: we don't need it shoehorned into every single release.