Serving barely useful GLM 5.2 costs what? $15k? Actually useful is like $50k? You’ll never recoup the cost unless you ‘locally’ means ‘inference provider is not the model provider’?
This is probably the dumbest possible way to do it. Just buy tokens through open router and you could run it all month 24/7 at 100tps for practically nothing. There are tons of ways to pay for things without giving your personal information.
$15k or $50k is pretty cheap all things considered (a year ago it would have been more expensive, one person can spend that in a month or two)
I bought my spark and the models have already improved in that time (qwen3.6, speculative decoding 2x tgen, diffusion gemma 4x tgen) and I expect this to improve. Look out another 2-3 years, local is going to be very competitive.
That ship has sailed. Even if you never even tab complete in cursor, if you don’t let LLMs review your code you’re very, very behind unless you’re in a deeply specialized domain which doesn’t have any public training data available. Anything remotely public and you’re just outpaced.
shrug Not really a me problem, but I'd counsel taking an afternoon to reflect on what part of any of this is actually inevitable. You know, maybe come up for air for a minute and examine the industry hype from 30,000 ft.
Ridiculous. Haven't you heard? All critical thinking skills have long since been sacrificed on the altars of the AI gods and it's inconceivable that we write any code the old way. If you actually understand your code it means you're a luddite and are going to be left behind. /s