[nightski]

AMD gatekeeps this functionality behind it's non-consumer cards. They don't realize that having a consumer card and being able to develop on it is a gateway to using AMD. I can use CUDA on any Nvidia card I buy. I can't believe they are so incredibly dense on this.

[65a]

You can run inference and training on consumer AMD cards today. It works fine, including llama.cpp, stable diffusion, hugging face transformers, etc. Way cheaper for a given performance/VRAM target as well.

[thrtythreeforty]

Maybe so. But it isn't confidence inspiring when I go to see which cards are supported and I see this issue:

https://github.com/ROCm/ROCm/issues/1714

With Nvidia cards, I know that if I buy any Nvidia card made in the last 10 years, CUDA code will run on it. Period. (Yes, different language levels require newer hardware, but Nvidia docs are quite clear about which CUDA versions require which silicon.) I have an AMD Zen3 APU with a tiny Vega in it; I ought to be able to mess around with HIP with ~zero fuss.

The will-they-won't-they and the rapidly dropped support is hurting the otherwise excellent ROCm and HIP projects. There is a huge API surface to implement and it looks like they're making rapid gains.

[Symmetry]

That's from 2022. AMDs move to start generally supporting consumer cards is very recent.

[echelon]

Where's the official show of support? I'll believe it when I see it.

[Symmetry]

They're listed as supported on their website and they work. I'm not sure what there is besides that.

https://rocm.docs.amd.com/en/latest/release/gpu_os_support.h...

You have to click on the "Radeon" tab for the commercial cards.

Yes, it's annoying that they only officially support Ubuntu 22.04 but it is official support and you can get other OSs and cards to work.

[anuraaga]

The article specifically is about AI. Don't most useful LLM models require too much RAM for consumer Nvidia cards and also often need those newer features, making it irrelevant that a G80 could run some sort of cuda code?

I'm not particularly optimistic that ecosystem support will ever pan out for AMD to be viable but this seems to be giving a bit too much credit to Nvidia for democratizing AI development, which is a stretch.

[nightski]

First of all, LLMs are not the only AI in existence. A lot of ML, stats, and compute can be run on consumer grade GPUs. There are plenty of problems that aren't even applicable with an LLM.

Second, you absolutely can run and fine tune many open source LLMs on one or more 3090s at a time..

But being able just to tinker, learn to write code, etc.. on a consumer GPU is a gateway to the more compute focused cards.

[zamalek]

There's a difference between officially supported, and supported. My 6900XT, an unsupported card, works just fine.

[thrtythreeforty]

Then they should indicate that! Putting me off from considering an AMD card for purchase is very detrimental to building a userbase.

[zamalek]

I 100% agree with that. The override envar (HSA_OVERRIDE_GFX_VERSION) is also buried deep in their documentation. NVIDIA is eating AMD's breakfast with GTX3060s while they are trying to peddle 7900XTs.

[65a]

Pretty sure my Radeon R9-285 would work if I force gfx802 offload arch when building for ROCm, but...what are you going to do with decade-old VRAM support? 2gb is not enough for anybody.

[Symmetry]

That's something that's started changing over the last few months. Official support for the RX 7900 GPUs for Linux has been added to the most recent versions of ROCm and over on the ROCm subreddit people are reporting success getting other RDNA 3 cards working. On Windows you've got consumer cards from the previous generation getting official support too.

This is, obviously, way overdue and it might not be enough to let AMD get back into the race but

[latchkey]

In my eyes, the real problem is that there is no cost effective developer access to high end cards, like the MI300x. This breaks the developer flywheel that you would normally point at consumer cards for.

Where can you rent time on one? Traditionally, AMD has only helped build super computers, like Frontier and El Capitan, out of these cards.

This time around Azure [0] and other CSP's (cloud service providers) are working to change that. I will have the best of the best of their cards/systems for rent soon.

[0] https://techcommunity.microsoft.com/t5/azure-high-performanc...

[LeanderK]

lol, this is so stupid. Don't they realise that people usually develop locally and train on a server? You don't need a super beefy GPUs to do it, so you buy nvidia. So people are used to nvidia, debug and fix bugs etc. It's not a very smart decision, looks like the decision makers have no idea what's going on.

[paulmd]

AMD’s opencl runtime also historically has an incredible number of bugs and paper features that make any sort of portability between nvidia and AMD cards quite difficult - you are running a special build and a different codepath for AMD anyway, there was no gain from using the ostensibly portable approach.

[latchkey]

The gain is that you can't find time on NVIDIA cards right now. Decentralizing away from a single provider lowers the risk on your business significantly.

Case in point, OpenAI closed new signups because they couldn't keep up with demand and they literally have all the resources in the world to make things happen.

[riffic]

*its

[joshstrange]

It's a possessive apostrophe: https://www.grammarly.com/blog/possessive-apostrophe/

[skyyler]

Did you read that article you linked?

>The important thing to remember is don’t use possessive apostrophes with any pronouns, either possessive pronouns or possessive adjectives.

>If you see an apostrophe with a pronoun, it must be part of a contraction.

>its—possessive adjective of it

>it’s—contraction for “it is”

"its" would be correct in the root comment.

[joshstrange]

No I didn't read it fully, I knew what the concept was called, found an article, skimmed it until I was sure it was talking about the thing I thought it was. I'm almost positive I was taught in school that "it's" is a valid possessive apostrophe case and honestly I think it's stupid that it's not. I find "its" more confusing personally.

[skyyler]

>I knew what the concept was called, found an article, skimmed it until I was sure it was talking about the thing I thought it was

You just managed to summarise why having conversations with strangers is so difficult on the internet these days.

Instead of considering that you were incorrect, even for a moment, you sought an article that you thought would confirm the ideas that you already had. Without even actually reading it, you used it as evidence that you were correct all along.

Even though the article very clearly illustrates that you were mistaken.

Fascinating.

[joshstrange]

You do understand that's not really what happened here right?

The concept IS called possessive apostrophe and literally until this minute I wasn't aware that "it's" isn't grammatically correct when the "it" in the sentence is being used possessively. I didn't just find an article I thought agreed with me and fire it off, I thought "it's" was valid and riffic didn't know about possessive apostrophes (which again, I had wrong in this case). I didn't look for "it's" in the article because that wasn't up for debate in my mind (again, I can't stress this enough, I was wrong about the usage, any comment I've ever made uses "it's" in this case because that's how I thought it was used), I was just looking for the concept as a whole to link. I got a minor piece wrong and you want to lump me in with everyone who picks the first article that "agrees" with them.

[riffic]

just sound it out in your head. If you see an apostrophe with it's, it usually means "it is"

Otherwise someone oopsed and you can leave a dumb grammar comment on HN

[HDThoreaun]

You’re thinking of its’

[riffic]

Please prove that is a valid construction.

[riffic]

that's nonstandard though, lol.

it's means it is or it has