[ragebol]

Probably saves a bit on the gas bill for heating too

[rglullis]

Snark aside, even in Germany (where electricity is very expensive) it is more economical to self host than to pay for a subscription to any of the commercial providers.

[CraigJPerry]

I don’t know, it’s kind of amazing how good the lighter weight self hosted models are now.

Given a 16gb system with cpu inference only, I’m hosting gemma2 9b at q8 for llm tasks and SDXL turbo for image work and besides the memory usage creeping up for a second or so while i invoke a prompt, they’re basically undetectable in the background.

[szundi]

If only we had heat-pump computers

[ragebol]

I'd gladly run whatever model you want at home, rent it out so you can pay for both heating, the GPU and the power consumed :-)