Hacker News new | ask | show | jobs
by ctoth 609 days ago
> It seems a little silly to pretend there’s a scaling “law” without plotting any points or doing a projection.

Isn't this Kaplan 2020 or Hoffmann 2022?

1 comments

Yes, those are scaling laws, but when we see vendors improving their models without increasing model size or training longer, they don't apply. There are apparently other ways to improve performance and we don't know the laws for those.

(Sometimes people track the learning curve for an industry in other ways, though.)