[nightpool]

Is it? When they could be having you running an even more powerful model on a $50k machine they sell by the pallet-load to enterprise consumers? We already see RAM manufacturers abandoning the low-end market in favor of server support. It's not clear to me that Nvidia sees personal GPUs as their best long term investment compared to selling millions of server-farm class machines

[layla5alive]

You mean a $500k machine, or a $15M rack... the costs have gotten unimaginably large from the lens of just a decade ago.

[anonzzzies]

... for AI... which people have to convince themselves they 'need'. I am a user, I have a 1m$ rack for my small company for AI, but I have currently 189 racks that cost about 15k one off per rack. And those make us a LOT more money than the AI ones ; most are departmental apps or websites that take very little processing power and are low on everything while people pay nice money to keep them going. Our support is better than the rest; you can just call me or come to my house and we'll do something. Not many clients leave, ever. For the past 30 years. I buy 5-10 year old servers which are complete overkill for 99% of my clients (some are fortune 500 companies but departmental stuff) and they just never leave. Makes vastly more money than AI and we are still cheaper than AWS and the department does not have to get a flexible budget (like they would have to if AWS) with us. We just have the same fixed price + yearly inflation correction and that's it. For 30 years.

[davnicwil]

Selling to individuals can be a hugely more robust predictable business, the problem with selling by the pallet load is spiky revenue that can also quickly fall off a cliff if larger customers stop buying. The other problem is sales negotiations driving down margins for bulk buyers etc. Consumer hardware is a very attractive market in lots of ways, just look at Apple.

[leoc]

Right. It's probably important to distinguish what the hardware manufacturers' incentives are when their supply is constrained and when it isn't. (I am not an expert on anything.) As long as supply is strongly constrained they're naturally going to sell to the highest bidders, which are the big LLM SaaS players. If and when supply is no longer constrained, though, things will look very different:

1) The LLM SaaS companies are a form of vertical disintegration for the hardware providers, a middleman covering costs and taking profits out of the money that comes from customers to the hardware providers. That changes somewhat if there are no longer good models available for local use at no cost to the hardware guys, but only somewhat

2) The LLM SaaS companies are efficient users of their hardware resources. While supply is constrained this helps to make them top bidders and so attractive customers for the hardware manufacturers. When supply is not constrained this should reverse. Which is the more attractive class of customer to a hardware maker: the company full of people with higher degrees who spend their whole working day fighting to pare back resource usage, or the guy who leaves his laptop idle about 18 hours per day on average?

It's notable that nVidia, for instance, has continued to put significant emphasis on AI compact desktops and laptops. And while no doubt that's partly in the service of better developer relations and good PR in general, it's probably also nVidia eyeing the exit, and preparing for a future transition from selling shovels to the army to selling shovels at Walmart. But of course the future isn't clear and obvious. If the hardware makers, maybe the RAM guys in particular, turn out to have underbuilt future capacity starting in the present then we could be stuck in constrained supply for quite a long time. (Futher) government action could affect things etc. etc. And if the frontier labs soon find new ways to use still larger amounts of memory, GPU capacity etc. that isn't butting up against diminishing returns then they'll likely remain kings for some time, though that does not seem probable now.