[cfiggers]

There's a survivorship bias going on here. You've never heard of the thousands of teams out there that are Mistral's size but AREN'T getting results that compete on the global stage, but they do exist. But you've heard of Google, whether they're getting it right or not.

[paxys]

"Thousands of teams" is a vast exaggeration. A tiny handful of companies out there have received funding to the tune of a billion dollars for model training like Mixtral. All of them have researchers with loaded resumes, and most are producing stuff of value. The thousands of other startups in the ecosystem are then taking these APIs and adding trivial abstractions on top.

[pixl97]

I wonder how much GPU compute time is effectively being discarded by these companies as the results turn out to be garbage?