[seydor]

It has been moved to hyper-scale engineering since a few years. The science of their engineering is still progressing (e.g LoRA is open science) , and it seems like whatever these companies are adding is not something fundamentally new (considering the success of LLaMa and the recent google memo that admits they have no moat).

And the various "Model cards" are not really in depth research but rather cursory looks at model outputs. Even the benchmarks are mostly based on standard tests designed for humans, which is not a valid way to evaluate an AI. In any case, these companies care more for the public perception of their model so they tended to release evaluations of its political-sensitivity. But that's not necessary the most interesting thing about those models nor particularly valuable science

[whimsicalism]

Your comment reads to me (someone in the field) like it is informed just by reading popular articles on the topic since 2022. The "Google memo" should basically have no impact on how you are thinking about these things, imo.

The field is taking massive steps backward in just the last year when it comes to open science.

> And the various "Model cards" are not really in depth research but rather cursory looks at model output

Because they are no longer releasing any details! Not because there hasn't been any progress in the last year.