[IAmNotACellist]

I deeply crave prosumer hardware that can sit on my shelf and handle massive models, like 200-400B at a reasonable quant. Something like Groq or Digits but at the cost of a high-end gaming PC, like $3k. This has to be a massive market, considering that even ancient Pascal-series GPUs that were once $50 are going for $500.

[darksaints]

I have that irresistible urge too, but I have to keep reminding myself that I could spend $2000 in credits over the course of a year, and get the performance and utility of a $40k server, with scalable capacity, and without any risk that that investment will be obsolete when Llama5 comes out.

[almostgotcaught]

> This has to be a massive market

It's not - it's absolutely a vanishingly small market.

[sofixa]

The Framework Desktop is one not absurdly expensive option. The memory speed isn't great (200 something GB/s), but any faster with those requirements at least doubles the price (e.g. a Mac Studio, only the highest tier M chips have faster memory).

[zozbot234]

> I deeply crave prosumer hardware that can sit on my shelf and handle massive models, like 200-400B at a reasonable quant.

So, an Apple Mac Studio?

[numa7numa7]

Nvidia's working on it. 200B at $3k

https://www.nvidia.com/en-us/products/workstations/dgx-spark...

[renewiltord]

At home people would rather use the cloud.