[adventured]

The training will always be extremely expensive and beyond the grasp of commodity computing at a low'ish cost.

The reason for that, is that the push will always be one direction in terms of size and improvement. That isn't going to stop. It will continue to push the edge of resources.

It's not Nvidia that's holding back the premise. AMD and Intel also can't do anything for you beyond what Nvidia can.

One might proclaim: yeah but in five years you'll be able to train CodeLlama 34B on a modest commodity desktop. Nobody will want to do that at that point, they'll want access to CodeLlama 204B.

The money will be in hosting these as a service business, and building further ecosystems around that. Not much different than the way a lot of major open source oriented companies have made their money, despite their core product being largely free to use.

[cbozeman]

I think you're correct in the "right now", but I think things fall off dramatically in the next 5-10 years. NVIDIA's progress from 2018 to 2023 was insanely impressive. AMD's was too in fact, at least technologically. Intel is third, but at least still in the game.

Where your premise here falls apart, is the age-old adage, "Don't let perfect be the enemy of the good."

We probably don't need CodeLlama 204B to be highly effective in our jobs as developers. Just like physicians probably don't need DocLlama 566B to be more effective in their jobs. We only need good enough. Good enough to be useful and insightful; not the All-Seeing Oracle that has insight beyond mortal man.

I think that revolution is probably coming sooner than we think. I'd argue that in 5-10 years, we'll have hardware powerful enough that a motivated and moderately well-heeled consumer can train their own LLM with domain-specific knowledge and have a product as useful as Llama8 1.3T or ChatGPT-14. There'll always be a market - I think anyway - for incredibly powerful general-purpose LLMs that are subscription-based (whatever that looks like in terms of cost / month), but locally trained, locally run LLMs designed to do one thing and do it well? I think that's the real money and the real future.

[a_wild_dandan]

Well said. Companies will want:

* Pricing stability -- your business costs cannot be at the whim/survival of some particular company. Competition (including local options) is the solution.

* Model stability –– model output must be reproducible. If it's shifting unreliably, that's a showstopper.

* Alignment –– The model I use must be aligned to my business, not to Some AI Business. I don't want a model that was kneecapped in some secret, arbitrary way that a group of largely white, male, West Coast techbros decided was crucial for my business.

* Privacy –– Sensitive company data NEVER leaves the building. It sure as hell doesn't get expedited to a company with strong incentives to hoover up and use that data.

There will soon be 3 big competitors racing each other in the GPU space. Specialized AI hardware will grow in volume and power. Algorithms will be optimized. Perhaps new algorithms will appear for distributed training of open source models by millions of consumers cards (reminiscent of the Folding@home project). Hardware 5 years from now will blow away current tech.

Intelligence is a widely applicable asset with strong incentives to pop up everywhere. It won't sit gated behind a handful of companies. That's true regardless of substrate -- proteins or silicon.