[suprjami]

So I can spend thousands of dollars to have an unstable training environment and inference performance worse than a US$200 3060.

Wow. Where do I sign up?

[vardump]

3060 doesn't have 128 GB RAM.

[moffkalast]

128GB / 12 GB = ~11, * 200€ = only 2200€ plus mining rig mobo.

It would be cheaper to buy up a dozen 3060s and build a custom PC around them than to buy the Spark.

[Mars008]

It will be really hard to put a dozen in a single PC. Then to connect them at good speed. Add to that a few new power lines to feed it all.

[moffkalast]

True, and overkill too. With DDR5 partial offloading it would only take probably two or three at most to outperform it in every metric except power draw. My point was more that the pricing is absurd for the performance.

[Mars008]

I agree on pricing. But it includes 'free' software. Similar model Apple has. You don't buy just hardware.

[pjmlp]

Except the Spark was designed to have everything nicely working.

[suprjami]

And as this post shows, it doesn't.

[pjmlp]

More than most AMD stuff.

[suprjami]

And a 14B model running at 22tg/s means you won't be using that 128G RAM for inference either.

[yunohn]

Yeah I’m honestly unclear on Nvidia’s thinking here - inference speed is unbelievably slow for the price.

Given the extreme advantage they have with CUDA and the whole AI/ML ecosystem, barely matching Apple’s M-ultra speeds is a choice…

[airspresso]

Definitely a choice to give it low memory bandwidth. Probably to avoid customers thinking it can replace any data center GPU for inference use-cases.

[kgeist]

You can use all 128 GB if you use a MoE model

[thehamkercat]

The memory bandwidth on this thing is absolute trash, better buy a mac mini/studio with this much ram if you're throwing this much money, it'll be faster (M4 Max)

[suprjami]

Agree, any Max or Ultra should walk all over this thing, and has the advantage of many years of already-working software.

Apple benchmarks: https://github.com/ggml-org/llama.cpp/discussions/4167

[bigyabai]

It really depends, the metrics are kinda all over the place right now: https://docs.google.com/spreadsheets/d/1SF1u0J2vJ-ou-R_Ry1JZ...

(cited from https://lmsys.org/blog/2025-10-13-nvidia-dgx-spark/)