Hacker News new | ask | show | jobs
by bckr 1153 days ago
The paper has a “preliminary scaling law” diagram. The shape of the graph is the same, but with 20% fewer FLOPS.

The real breakthrough is that Hyena apparently has an unlimited context window.

1 comments

>The real breakthrough is that Hyena apparently has an unlimited context window.

It's extrapolated volition time (゚∀゚)

Is it? Removing the context window limit is big, no doubt, but inference still takes time (and compute).
I think GP is talking about the ability of an AI to make decisions with reference to context from the past and therefore have a “will extended over time”