[echelon]

Stop trying to run them locally, folks.

You don't own your fiber connection. So why try to own another rapidly depreciating, expensive, and annoying asset?

Rent cloud GPUs!

You get to participate in the ownership, data control, price control, and hacking culture without having to Frankenstein some hobbyist box that costs a ton, is distilled down to functional uselessness, and is a PITA to maintain.

[satvikpendem]

If I'm gonna rent cloud GPUs I might as well just use a subsidized cloud agent like Claude or Codex. As for depreciation, that is true, but the bet is that models get better for a certain parameter count faster than your hardware becomes obsolete, such as Gemma models for example at the same 30 billion parameter count being much better than some years ago.

[gizajob]

This comment is like the antithesis of hacker culture. Can’t tell if being ironic or not.

[echelon]

Lobotomized RTX models are playthings.

People building this stuff are "year of linux on desktop"ing open weights AI. It's a huge opportunity cost - not just for you, but for the open source community at large.

You need to double down on big fat honking models that take multiple H200s to run. That's where the real power lies, and that's where our entire community needs to focus our efforts if we want to keep the delta between frontier and the proletariat small.

The more we build for people and enterprises to run big weights in private clouds, the better. That's the real treat to Google, Anthropic, and OpenAI. Your RTX cards don't make a dent in the death star.

[gizajob]

You could be an IBM executive writing about the Apple I.

[echelon]

There's a world of difference between the Apple I and setting up CUDA drivers and Python.

You're in a small community of hobbyists. Cheap hobbyists who mostly don't pay for the stuff.

It's a bad growth market. It's a bad space to develop products. And it's as distracting to brilliant minds as bitcoin.

It's a suboptimal nerd snipe, and all the effort spent there is effort not being spent building actual frontier capabilities in the open.

[AdieuToLogic]

> Stop trying to run them locally, folks.

"Locally" is a relative qualifier if one defines locality as not being reliant on a SaaS vendor. IOW, locality does not necessarily imply execution on machines specifically owned/operated by an organization.

> Rent cloud GPUs!

This would qualify as "locally" in the above definition. There is also a case to be made that h/w ownership (GPUs included) and operation can result in a net cost reduction for some use-cases.

However, where exposing intellectual property results in regulatory violations and/or undue legal exposure, running models "locally" is not only a good option, it is the only option.

[broadsidepicnic]

> You don't own your fiber connection. So why try to own another rapidly depreciating, expensive, and annoying asset?

Single mode fiber can serve for tens of years without problems and push the fastest speeds available today. I do not understand this comparison.

[echelon]

We don't need to own the hardware.

We need to own the software and the models.

Playing around with local models is like playing around with Ubuntu and Arch in the 00's. It's a fun toy, but it doesn't make a big economic dent, and it doesn't ensure we retain our rights and a slim capability gap against the frontier.

Developing software that works with big models, showing up with economic demand - that ensures that capability gets built and that open whittles away at closed at the very frontier.

More customers going to tiny hobbyist models also sucks oxygen out of the room for more large scale open models. We need to put economic demand on the larger open weights.

I see two futures ahead:

Future 1 - Big companies alone have access to the most productive models. Consumers play with API offerings and tiny RTX-scale models that lack the same capability.

Future 2 - A robust assortment of open weights models keep a very slim capability gap against the most mature frontier models. There's a viable economy around using and supporting these big models. Prosumers and enterprise can easily rent spot instances and spin up weights on-demand for a variety of tasks. There are rich model and fine tune marketplaces, a wide assortment of tools that can call these models, and easy tools to train models for any task from any foundation pretrain starting point.

I'd rather we go down path #2.

[irishcoffee]

> You don't own your fiber connection. So why try to own another rapidly depreciating, expensive, and annoying asset?

Like a car? Because I don’t want to depend on uber or a taxi service.

What does fiber have to do with anything? I don’t need the internet to run my local models.

[pbgcp2026]

Yes, "Like a car". LOL. You realise that many people in Europe and Asia do not own a car at all? Public transport, eBike / scooter, Tuk-Tuk, walk.

The local LLM "privacy" war had been already lost.

[gizajob]

You realise how many people in Europe do own a car?

And how exactly has the privacy war been lost?

[pbgcp2026]

The privacy war has been lost in two ways (at least) 1) Running locally lobotomised models makes no sense; 2) as someone said here, the Gov will declare local AI a felony. And they will enforce it. So those "many people" will buy V8 cars limited to V4 and declared illegal to drive without registration and license, even locally in your own yard, and they may go to jail if they attempt to activate other 4 cylinders. Oh, wait ... isn't it how car laws work now? ;-)

[gizajob]

Ludicrously paranoid take.

[pbgcp2026]

I'm glad that you saw my point and I apologise for stepping on your ego. :-)