[SwellJoe]

Ah, that makes all the difference. Being normal has a lot of value. I mostly take it back. I still don't like the custom Ubuntu Nvidia ships, but if the system is normal any Linux distro for ARM works, it becomes an entirely viable thing.

[verdverm]

Nvidia didn't make a great out-of-box experience for sure, one of the first things I do on my Spark is set init 3 and disable bluetooth. Docker was working on one, not the second. Many typical local tools used in Ai were not installed (ollama,vllm,llama.cpp... all MIA)

[cmrdporcupine]

I mean, I never expected to have pre-packaged versions of any of those things that wouldn't just fall out of sync with upstream immediately anyways?

I basically treated it as a stock Ubuntu machine but on Aarch64, and ignored the stuff they installed. And then went looking for docker images for vllm that made sense and were up to date and hopefully tuned for NVFP4 on the hardware and...

... that was more the disappointing part.

[SwellJoe]

Yeah, the stuff they preinstall is crazy. It's a ~16GB OS image, which is a huge pain in the ass (in addition to all the other things that are a pain in the ass about Nvidia's OS situation); I can't ship their 16GB pile of crap plus our custom 3GB of stuff on my robots, there's no reasonable way to do OTA upgrades with a pile of shit like that. It's just an unpleasant experience all around. What I always want is a nice clean basic install, and then my custom packages of everything I need goes in. Especially for embedded cases, like the Jetson line is allegedly intended for, their whole approach is just stupid, and they don't provide any documentation for how to not use their stupid thing.

Still, I'd rather have the blazing prefill speed of the Nvidia over my Strix Halo, but I wasn't willing to spend nearly twice as much for it (when I bought, the Strix Halo was $2100 and the GX10 was going for $3700, I think). Now that the difference between the two is much smaller, there's no reason to get the AMD.

[cmrdporcupine]

The $2100 vs $3700 price difference really came down to the ConnectX hardware in the thing, I guess. But the price gap narrowed over time, now they both suck.

I am not sure why NVIDIA couldn't have released a cheaper version of the thing with just standard Ethernet on it and leave it at that. Hardly anybody can afford two or more of them to cluster via ConnectX anyways.

[verdverm]

The Spark does a system update on first boot, so they would have updated had they been installed

I'm running vllm + GoModel on k3s, works like a charm. Even wrote some CUE last night to generate the values file for my Helm chart. It calculates the GPU percentage for me from the memGB I assign to a model.