[aspenmartin]

Your observations are right but pretty insane to consider them a pure PR company lol. They are making more frequent releases so yes the release-to-release quality is smaller but we’re still ascending quality and reliability curves the same way we have since GPT-3. You get a GPT4->5 leap every like 17 or 18 months I think it is

[kingkongjaffa]

The gradient of improvement is absolutely not the same.

[aspenmartin]

If anything its slightly higher. Feel free to provide any evidence to the contrary.

ECI (good aggregate measure using IRT): https://epoch.ai/eci?view=graph&tab=release-date&subset-view...

METR time horizon (now topped out): https://metr.org/time-horizons/

[WASDx]

I like this one, although its data seem to overlap with ECI.

https://artificialanalysis.ai/trends