[barrkel]

People spend a lot more than that on a car they use less, especially if they're in tech.

The RAM choice was because I have never regretted buying more RAM - it's practically always a better trade than a slightly faster CPU - and 96GB DIMMs were at a sweet spot compared to 128GB DIMMs.

That, and the ability to have big LLMs in memory, for some local inference, even if it's slow mixed CPU/GPU inference, or paged on demand. And if not for big LLMs, then to keep models cached for quick swapping.

[sfn42]

I bought a 4 year old car for significantly less than that. And I can get a computer that can do 99% of what your monster can do for like 10% of the price. And if I want LLM inference I can get that for like $20 a month or whatever.

I don't mean to judge, it's your money but to me it seems like an enormous waste. Just like spending $100k on a car when you can get one for $15k that does pretty much exactly the same job.

[barrkel]

Sure. You're right, it is my money. And I pay even more for inference on top; I have OpenRouter credits, OpenAI subscription, Claude Max subscription.

It's not so easy to get nice second-hand hardware here in Switzerland, and my HEDT is nice and quiet, doesn't need to be rack-mounted, plugs straight into the wall. I keep it in the basement next to the internet router anyway.

The "sensible" choice is to rent. It's the same with cars; most people these days lease (about 50% of new cars in CH, which will be a majority if you compare it with auto loan and cash purchase).

[sfn42]

I don't think leasing cars is sensible. Last time I checked, for cheaper cars mind you, I would essentially pay 60% of the sticker price over a few years and then not have a car at the end of it. Would be better to buy a new car and then sell it after the same time. But what's even better is to not buy a new car, let some other sucker take the huge value loss and then snatch it up at a 30-60% discount a few years later. Then you can sell it a few years after that for not much less than you paid for it. I've had mine a year and right now they're going for more than I paid.

I think leasing might be okayish if you find a really good deal, but it's really not much different than buying new which is just a shit deal no matter how you turn it. A 1-4 year old car is pretty much new anyway, I don't see any reason to buy brand new.