[didgetmaster]

Maybe I am just a natural skeptic, but whenever I see a headline that says 'method x reduces y by z%'; but when you read the text it instead says that optimizing some step 'could potentially reduce y by up to z%'; I am suspicious.

Why not publish some actual benchmarks that prove your claim in even a few special cases?

[dragonwriter]

Well, one, because the headline isn't from the researchers, its from a popular press report (not even the one posted here, originally, this is secondary reporting of another popular press piece) and isn't what the paper claims so it would be odd for the paper's authors to conduct benchmarks to justify it. (And, no, even the "up to 95%" isn't from the paper, the cost savings are cited per operation depending on operation and the precision the operation is conducted at, are as high as 97.3%, are based on research already done establishing the energy cost of math operations on modern compute hardware, but no end-to-end cost savings claim is made.)

And, two, because the actual energy cost savings claimed aren't even the experimental question -- the energy cost differences between various operations on modern hardware have been established in other research, the experimental issue here was whether the mathematical technique that enables using the lower energy cost operations performs competitively on output quality with existing implementations when substituted in for LLM inference.

[baq]

OTOH you have a living proof that an amazingly huge neural network can work on 20W of power, so expecting multiple orders of magnitude in power consumption reduction is not unreasonable.

[etcd]

Mitochondria are all you need.

Should be able to go more efficient as the brain has other constraints such as working at 36.7 degrees C etc.

[andrewstuart]

https://github.com/microsoft/BitNet

"The first release of bitnet.cpp is to support inference on CPUs. bitnet.cpp achieves speedups of 1.37x to 5.07x on ARM CPUs, with larger models experiencing greater performance gains. Additionally, it reduces energy consumption by 55.4% to 70.0%, further boosting overall efficiency. On x86 CPUs, speedups range from 2.37x to 6.17x with energy reductions between 71.9% to 82.2%. Furthermore, bitnet.cpp can run a 100B BitNet b1.58 model on a single CPU, achieving speeds comparable to human reading (5-7 tokens per second), significantly enhancing the potential for running LLMs on local devices. More details will be provided soon."

[jdiez17]

Damn. Seems almost too good to be true. Let’s see where this goes in two weeks.

[andrewstuart]

Intel and AMD will be extremely happy.

Nvidia will be very unhappy.

[l11r]

Their GPU will still be needed to do training. As far as I understand this will improve only interference performance and efficiency.

[TheRealPomax]

Because as disappointing as modern life is, you need clickbait headlines to drive traffic. You did the right thing by reading the article though, that's where the information is, not the title.

[phtrivier]

Fair enough, but then I want a way to penalize publishers for abusing clickbait. There is no "unread" button, and there is no way to unsubscribe to advertisement-based sites.

Even on sites that have a "Like / Don't like" button, my understanding is that clicking "Don't like" is a form of "engagement", that the suggestion algorithm are going to reward.

Give me a button that says "this article was a scam", and have the publisher give the advertisement money back. Of better yet, give the advertisement money to charity / public services / whatever.

Take a cut of the money being transfered, charge the publishers for being able to get a "clickbait free" green mark if they implement the scheme.

Track the kind of articles that generate the most clickbait-angry comment. Sell back the data.

There might a business model.

[NineStarPoint]

I doubt there’s a business model there because who is going to opt in to a scheme that loses them money?

What could work is social media giving people an easy button to block links to specific websites from appearing in their feed, or something along those lines. It’s a nice user feature, and having every clickbait article be a chance someone will choose to never see your website again could actually reign in some of the nonsense.

[phtrivier]

> I doubt there’s a business model there because who is going to opt in to a scheme that loses them money?

Agreed, of course.

In a reasonable world, that could be considered part of the basic, law mandated requirements. It would be blurry and subject to interpretation to decide what is clickbait or not, just like libel or defamation - good thing we're only a few hundred years away from someone reinventing a device to handle that, called "independent judges".

In the meantime, I suppose you would have to bring some "unreasonable" thing to it, like "brands like to have green logos on their sites to brag" ?

> What could work is social media giving people an easy button to block links to specific websites from appearing in their feed, or something along those lines.

I completely agree. It's a feature they have had the technology to implement such a thing since forever, and they've decided against it since forever.

However I wonder if that's something a browser extension could handle ? A merge of AdBlock and "saved you a click" that displays the "boring" content of the link when you hoveron a clickbaity link ?

[keybored]

Headlines: what can they do, they need that for the traffic

Reader: do the moral thing and read the article, not just the title

How is that balanced.