[SepiaSapient]

I'm simply a postgrad from a no name university, but I'm not sure that the future frontier will come from the current approach from OpenAI and Anthropic. Distilling just seems like another avenue to collect useful data, like using books or scrapping the human-net, not necessarily copycat behavior. Chucking another 1000 TWh and scaling with already pillaged human-output is having diminishing results.

The next "frontier" as in, an order of magnitude higher model capability, might come from eschewing the bitter lesson and trying to be clever instead of pilling on GPUs. (I'm sure US labs also try clever new architectures) Maybe the plateau is permanent in the medium term. The frontier will be the exercise in taking the 10T monster models into something that can be run locally with minimal degradation.

That jibes a lot more with "AI is just new tech" attitude than the (genuine or otherwise) "we will build the Godhead in our image".