[sdenton4]

I got a very different message from this, actually much closer to the problem of incumbent advantage.

The known-good thing has been heavily optimized for performance, making it much harder for new technologies to prove that they are better. This is similar to the problem of gas vs electric engines - we had a century of optimization and ecosystem development around gas engines, which creates an uphill battle for electric motors even though they are (eventually) superior on every way /except/ having that massive ecosystem.

The problem isn't as bad here, because software is much more flexible than hardware, and scaling laws give a reasonable way to try things out at smaller scale before going whole hog.

[lukeschlather]

what is the known-good thing? The whole point is that LLMs were not optimized at all, they got better results than older ML algorithms just because they are able to use all of the GPU, where older algorithms are designed for 10yo GPUs and can't make use of modern GPUs. But now you do in fact have to optimize, to the point that transformers look a lot more complicated than "attention is all you need."

[RugnirViking]

the known-good thing is whatever was previously used for the application you're using llms for in any given case. This is often the structured application of human intelligence.