[aspenmartin]

We have very stable trends in observed performance in a LOT of different measurements and stable scaling laws that continue to hold true across at the largest scales we have built. These scaling laws have a reasonable theoretical justification and make predictions which continue to hold true. Pretraining perplexity does seem related to downstream measures of performance but those too show very stable trends in performance gains as well, even if you look at these in isolation. Look at the epoch capability index as a good summary statistic across a number of benchmarks.

So: yes we can and do make predictions with it and that’s how we get funding internally and externally to build at these scales.

Bottlenecks arise about 3 orders of magnitude from now.

On the other hand, any particular justification you have in mind for your point?

[t43562]

> So: yes we can and do make predictions with it and that’s how we get funding internally and externally to build at these scales