[ToucanLoucan]

AI boosters cling to this notion because it's the only way the massive data center buildouts make any sense at all. I guess you could say the US is winning the frontier AI race. Okay. I'm never going to grant a cloud service access to all the contents of my hard drive, that's just never going to happen, so if you expect me and a lot of people like me who feel similarly to get on this train, you better have a local, lightweight model too or we're not even having a discussion, the answer is just no.

[Our_Benefactors]

The thing is, frontier model providers don’t take your feelings into account even a little bit. It’s totally irrelevant to the discussion about the service they can provide, because that service is predicated on access to high power GPU slices that local models can’t touch. Those providers won’t be in an existential crisis because some people choose the privacy route, it’s a cost of doing business.

[ToucanLoucan]

Right but that service being sold is predicated on products being sold to users, yes? Or are we still pretending that the hyperscalers can just pass the same $20 billion between themselves and that's going to be a growth industry forever?

[ElevenLathe]

I suppose its possible that all the value to pay back the datacenter construction can be squeezed out of enterprise contracts where your employer can assent on the privacy questions, probably with some kind of complicated contract and insurance regime regulating things.

Even if so, if China is coming behind 6 months later selling laptops with hyper-efficient local models that are 80% as good as "frontier" ones, I imagine they'll get the consumer business AND a fair share of the enterprise business as IT managers look at their options during the next refresh cycle.

Given economies of scale, I think it's ultimately inevitable that the enterprise more-or-less follows the consumer on this, and the consumer is going to prefer local models. There's no ongoing cost after the initial purchase, and your data at least nominally stays within your control.

[ToucanLoucan]

I'm inclined to agree. The business world itself, and frankly you could make this argument about the entire AI industry as it were, runs on "fine" and 80% capable can probably get you there. And it's arguably even better for the hyperscalers since then, ongoing costs of AI users are basically nil. You can still have your massive datacenters and just keep them for tasks complex enough that they're actually worth spooling up.

Like I don't need an H100 or a dozen to summarize a PDF. And that's most of what I use AI for.

[Our_Benefactors]

If we are betting on which is an easier sale, $20-100 a month w/tech support included vs $5k-10k and a requirement for moderate technical ability, I would invest in the former not the latter being the proposition that drives the conversation about AI use.