[majormajor]

Running local applications is less efficient than thin clients to the cloud generally, not just in LLMs. The trick is that you can get to the point where it's effective enough, and affordable enough, that the control and availability factors become dominant.

[stingraycharles]

My point is that you will always get much more value / $ by using cloud based solutions.

[majormajor]

I just don't see how that's different from getting more value by giving all your employees the most stripped-down chromebook-type devices and running everything else in the cloud, than by giving them "proper" laptops with local apps.

It's a measure of a very thin sort of "value/$" that excludes a lot of other things that could be of value to a business, like control, predictability, and availability.

Thin clients have been going away for a long time. The trend has been to continue to push higher levels of compute into ever-smaller and ever-more-portable devices.

[sroerick]

I don't know that this is true. The cloud companies are making money, and inferrence is kind of just "hosting an inferrence server and trying to keep it humming 24/7"

But in many cases self hosted or dedicated boxes are cheaper than cloud.