[throwway120385]

Nvidia's software support remains "best in class" so they've become the default for a lot of this stuff. To overcome that I think you'd have to get a lot of hobbyists and by extension interested young people using your hardware instead of Nvidia. So you'd have to have good software support, decent hardware, and low hardware cost. People started with Nvidia because most machines had an Nvidia card in them, and Nvidia's software support was carried-over momentum from how they've supported their GPUs with PC gaming. I've tried AMD, for example, and the compatibility matrices and bugs and just general lacking software support for ML on their hardware makes it a total non-starter. It may be a lot better with current-generation stuff but it's going to take a lot for me to trust AMD for ML.

[Teever]

This should have been obvious to me before but for whatever reason reading your comment made me realize something.

The hardware that nvidia is selling can be used against them because it can generate the code that is lacking for hardware from other companies.

So there’s a world out there where Intel or AMD just generate the needed software with compute from rented Nvidia hardware or the end consumer does the same, akin to opensource volunteers spending their free time reverse engineering undocumented hardware for the last few decades.

That’s if the claim that other companies produce acceptable hardware that is mostly just kneecapped by poor software.

If that’s the case there’s a distinct chance that we’ll see feature parity between the different vendors soon enough.

Unless the models that run on nvidia hardware are inadequate for this task, but that sort of raises a catch-22 — if the models aren’t good enough to generate drivers and CUDA type software what are they good enough for?

[minraws]

True, Given I am young, hobbyist, and have written a lot of AI libraries for myself and small companies for local deployments, and can't afford any serious Nvidia hardware I will switch to whoever can get me a system with enough VRAM for a decent price under 1000$...

My best GPU is 5070Ti on which the best model I can run is Laguna S 2.1 118B with NVFP4 and offloading most experts on CPU with an expert router layer and 128k context.

I could kill for a 1000$ AMD or Intel card with 32 or 48GB of VRAM even GDDR6 around a 1000$ range but it feels like no one even cares.

With ~4 48GB cards I could seriously locally deploy most open models models with some SSD offloading and good 4bit quantizations with ok context sizes. But at current prices.. sigh... I tried begging Intel to maybe it's cards available in my region but alas no such luck especially not at reasonable prices.

Honestly getting an AI subscription feels cheap atm and working hard at solving and home labing stuff is insane.

Even in research no one cares about anything but Nvidia, because at this point they are expensive but they seem to care, I just don't see others caring though I have worked with Intel gpu and compute teams on implementing this stuff and they are very enthusiastic but well the companies themselves aren't solving anything for me as an individual, but they are begging for me to add support for their stack in software I maintain. The Irony is unreal, why should I even add support when no one can even use it.

Only company that had decent stuff was surprisingly Apple but their stuff is now too expensive as well. .sigh.

[ndriscoll]

You can buy an Arc Pro B70 with 32 GB of GDDR6 for $1k right now. Intel made exactly the thing you want.

[caycep]

It's sad that this is true and yet their software is filled with bugs...