Hacker News new | ask | show | jobs
by hirako2000 274 days ago
It solely is improving on efficiency. While it is extremely valuable given the disproportionate (to value) costs of these things, your statement almost sounds like it has improved an even more challenging aspect, pushing performance.
4 comments
embedding-shape 274 days ago
It's a generic comment that I don't think is even specifically about Kimi Linear or this submission, you could leave the same comment on almost any AI/ML submission and it'd say the same amount and be as relevant/irrelevant.
link
hirako2000 274 days ago
Agreed it would apply to any method or system that improved on efficiency. It doesn't diminish the feat. Not trying to minimize the impact of Kimi linear's gain. It is a novel and outstanding benefit applicable to LLMs.
link
acuozzo 274 days ago
> It solely is improving on efficiency.

Consider the implications of increases in efficiency *when you hold compute constant*.

The win is far more obvious when it's "we can do more with what we have" instead of "we can do the same with less".

link
naasking 274 days ago
There has been work that has pushed performance too, like tiny recursive models. Applying LLMs in recursive loops also improves output, so efficiency improvements make this viable, which can count as improvements in performance.
link
giancarlostoro 274 days ago
I've uh said this a few times. But AI is a bunch of people overpaying CS students to implement old algorithms and then realizing that they need Software Engineers to optimize the existing known systems. Most of AI (if not ALL of it) as we know it today has been coded for decades, we just never had the hardware for it.

A lot of the optimizations are not some ground breaking new way to program, they're known techniques to any Software Engineer or Systems Engineer.

link
embedding-shape 274 days ago
> A lot of the optimizations are not some ground breaking new way to program

Hindsight is a bitch huh? Everything looks simple now once people proved it kind of works, but I think you over-simplify "how easy it is".

Lots of stuff in ML, particularly recent ~5 years or so, haven't been "implementing old algorithms" although of course everything is based on the research that happened in the past, we're standing on the shoulders of giants and all that.

link