[mechagodzilla]

It does seem like it will be very, very hard for the companies training their own models to recoup their investment when the capabilities of open-weight models catch up so quickly - general purpose LLMs just seem destined to be a cheap commodity.

[jsheard]

Well, the companies releasing open weights also need to recoup their investments at some point, they can't coast on VC hype forever. Huge models don't grow on trees.

[mechagodzilla]

Or, like Meta, they make their money elsewhere and just seem interested in wrecking the economics of LLMs. As soon as an open-weight model is released, it basically sets a global floor that says "Models with similar or worse performance effectively have zero value," and that floor has been rising incredibly quickly. I'd be surprised if the vast, vast majority of queries ChatGPT gets couldn't get equivalently good results from llama3/deepseek/qwen/mistral models, even for those paying for the pro versions.

[riku_iki]

I think Meta folks just don't know how to come to this market and build something potentially profitable, and doing random stuff, because need to report some results to management.

[Philpax]

Eh, to some extent - there's still a pretty significant cost to actually running inference for those models. For example, no consumer can run DeepSeek v3/r1 - that requires tens, possibly hundreds, of thousands of dollars of hardware to run.

There's still room for other models, especially if they have different performance characteristics that make them suitable to run under consumer constraints. Mistral has been doing quite well here.

[mechagodzilla]

If you don't need to pay for the model development costs, I think running inference will just be driven down to the underlying cloud computing costs. The actual requirement to passably (~4-bit quantization) run Deepseek v3/r1 at home is really just having 512GB or so of RAM - I bought a used dual-socket xeon for $2k that has 768GB of RAM, and can run Deepseek R1 at 1-1.5 tokens/sec, which is perfectly usable for "ask a complicated question, come back an hour or so later and check on the result".