[DwarvenEngineer]

Ironically for home-lab users (ie those who primarily have access to consumer hardware), apple silicon is probably the best value (and also the most obtainable for running some of the smaller sized models. You can get unified memory of up to 192 GB in the mac studios. The equivalent for the "open market" is probably AMD's halo strix which offers up to 128GB unified memory, and the speed of the memory is much slower.

I guess the Nvidia Jetson AGX Thor also exists, but good luck getting your hands on that... and regardless that's also 128GB.

Put it this way, with 192 GB VRAM you can run deepseekv4... with 128gb only with lots of quantization

Edit: WTH! I was curious and i went to apple's website to price out a high end unit just to price comparison, and it looks like the high ram units are gone now, and they max out at 96GB?? Guess i missed that...

[freediddy]

Not supporting FP8 is a pain in the butt though. It feels like there are more compromises using Apple Silicon than just going with nVidia and CUDA.