[benlivengood]

Electricity is more expensive at home than where data centers are built, batch inference is more efficient at GPU/TPU inference per watt, power supplies in data centers are more efficient than in average consumer devices, entire racks can be fully powered off when not in use vs. standby power consumption, and of course the investment in hardware is amortized across many users in data centers. It allows more people to have access to larger models than everyone buying an M3 Ultra.

The economy of scale that data centers have is actually a good thing economically and environmentally for many kinds of demand.

I think that the most capable models will continue to be in high demand across the market until at least "a datacenter of PhDs" level of capability. At that point I can see a transition to more local model use if affordable consumer hardware is available (for the median human on Earth). If that turns out to be true then the hyperscaling will plateau at the level allowing sustained commercial/industrial "PhD"-level demand which we aren't at yet (all providers are still struggling to meet current demands).