[mapontosevenths]

If you bought it for inference you made a mistake. They aren't good at that. Use it to train models and experiment with ML. It's much better at that.

If you just want local inference buy a Mac.

If you bought early on, like I did, the Spark is probably worth double what you payed now. I think I paid $3,000 retail for mine and the last time I looked they were fetching close to $6k on ebay. I'm not sure if that's still the case, but you can buy a very nice Mac with $6k.

[trentor]

Mhm... maybe read my full comment?

[mapontosevenths]

Are you saying that it's slow for training? Sorry, your comment is confusingly worded to me.

I've not had any issues in that regard, but I'm working with LLM's not training diffusion models. Are you following one of the Nvidia provided recipes or inventing something on your own? The last time I looked into it they benchmarked very well, but we both know that doesn't always mean much.

[trentor]

Yes it is slow for training/fine-tuning decently sized LLMs and unusable for diffusion models. What are you training were you get usable speeds?