[segmondy]

The same way you use a cloud LLM.

[oblio]

I think the point was that for example for programming, people perceive state of the art LLMs as being net positive contributors, at least for mainstream programming languages and tasks, and I guess local LLMs aren't net positive contributors (i.e. an experienced programmer can build the same thing at least as fast when using an LLM).

[segmondy]

I know this is false, DeepSeekv3.1, GLM4.5, KimiK2-0905, Qwen-235B are all solid open models. Last night, I vibed rough 1300 lines of C server code in about an hour. 0 compilation error, ran without errors and got the job done. I want to meet this experienced programmer that can knock out 1300 lines of C code in an hour.

[drusepth]

Are 235B models classified as local LLMs? I guess they probably are, but others in this thread are probably looking more toward 20B-30B models and sizes that generally fit on the RAM you'd expect in average or slightly-higher-end hardware.

My beefy 3D gamedev workstation with a 4090 and 128GB RAM can't even run a 235B model unless it's extremely quantized (and even then, only at like single-digit tokens/minute).

[codazoda]

How much machine do you have to be able to run Qwen-235B locally?

[oblio]

Can you run any of those models without $20 000 worth of hardware that uses as much power and makes as much noise as a small factory?

[segmondy]

I run them with under $3,000 hardware and inference is about 500-600watts with no noise.

[nomel]

Without knowing what you were doing with that 1300 lines of code, there's not much insight that can be had from this.

[brookst]

I’m a mediocre C programmer on my best day and I assure you a highly competent programmer could probably use 200 lines of code to do what I achieve in 1300.

Just counting lines is not a good proxy for how much effort it would take a good programmer.

(And I am 100% pro LLM coding, just saying this isn’t a great argument)