[_jab]

I think the reason the goalposts keep shifting is really that these AI labs are illustrating Goodhart's law in action: https://en.wikipedia.org/wiki/Goodhart%27s_law

We're at a point where AI labs are fairly reliably able to create AI systems that can achieve good performance on given benchmarks, via targeted training efforts. You argue that it's not reasonable to expect that performance to generalize across other domains. But given that these same companies are trumpeting that AGI is around the corner, I think that's a fair expectation for us to have, and AI has disappointed in that regard.