[cmrdporcupine]

I have an NVIDIA Spark thingy (the ASUS one) and it's the same problem there, though the prefill side is superior to the Ryzen ones.

But I actually think 128GB is too little. There are some compelling models that are above what can fit in that at reasonable quants (e.g. DeepSeek V4 Flash) but could if the system was 256GB.

If RAM prices weren't so f*cked I think we'd be seeing 256GB and even 512GB unified memory systems becoming quite common. As it is I think it will be 10 years before >128GB becomes feasible on a regular consumer machine for normal people again.

[terhechte]

You can run DeepSeek Flash just fine on 128GB using lower quants. Antirez' DwarfStar supports it really well. The bigger the model, it seems, the less it is affected by high quantization. I know some people are working on quantizing GLM 5.2 so it can fit into 128GB (using additional tricks to select only some experts, etc). There's a lot going on.

[andy99]

> RAM prices weren't so f*cked I think we'd be seeing 256GB and even 512GB unified memory systems becoming quite common

I agree but unfortunately that speaks to the depth of demand right now. Even as more capacity comes online (if it does) I suspect it will all get used up without prices needing to drop. Hopefully on the other side of this 10 years from now it means RAM will be insanely abundant and we’ll all have machines with a couple TB as the same inflation adjusted price as say 16GB a few years ago. Considering single digit MB of RAM was the norm in the early-mid ‘90s its not so far fetched.