[woooooo]

Nvidia was pushing CUDA forward for over a decade before it started getting serious commercial traction. It's not like they blocked anyone else from developing viable GPGPU tech, they were just the only ones pushing it.

For like 8 years their drivers on Linux were a nightmare and AMD could have come in and done better.

[shmerl]

> For like 8 years their drivers on Linux were a nightmare and AMD could have come in and done better.

AMD eventually did while Nvidia's drivers remained a nightmare almost until these days. But sure, AMD could have done it sooner.

[paulmd]

> AMD eventually did while Nvidia's drivers remained a nightmare

and yet that trillion-dollar valuation built over the last decade is built with customers almost entirely running on those "nightmare" linux drivers, while AMD's linux drivers crash running the sample app on supported hardware+OS, and nobody at AMD cared until finally a tech-bro with a loud enough platform shamed them into fixing it...

... and this is something like AMD's third crack at the apple, and the first three sets of drivers (one of which is literally a Vulkan-branded spec) are just as non-functional today as rocm was a year ago.

(OpenCL, Fusion HSA/AMD APP, Vulkan Compute/SPIR-V... all still broken so badly that Octane called them out for being unable to successfully compile their renderer and for lack of vendor support, so badly that Blender pulled support after years of turbulent and poorly-performing attempts to work with AMD, etc)

[shmerl]

Nvidia only cares about a specific market. I.e. it doesn't care about desktop users. That's what I was talking about. So despite their pools of cash, Nvidia is a trash company when it comes to Linux support.

[wmf]

AFAIK a lot of Hollywood visual effects are done on Linux + Nvidia so they probably support that market.

[shmerl]

Not really familiar with it, but hopefully they can get unstuck from Nvidia, especially on Linux. Only very recently things started improving it seems and not even with Nvidia's effort but outside community working on nova + nvk.

[p_l]

nvidia linux drivers efforts were pretty much always driven by that market, it's not getting unstuck because it's supported well.

Hell, nvidia drivers might be often complained about, but for years I would take nvidia because the crappiness was manageable and close to nothing if you were in the target market (desktop workstations running X11 on only nvidia GPUs? The only issue was if you were running super latest kernel).

[CoolGuySteve]

AMD and Apple tried to push OpenCL but the design of it, a C-like kernel compiled to the GPU with LLVM and managed by the Khronos consortium, tended to lag in absolute performance to CUDA which was able to take advantage of evolutions in GPU design more closely.

Nowadays almost nobody cares about OpenCL.

[jjoonathan]

The feature lag wasn't the problem, the bugs were the problem: the only reliable OpenCL implementation was the one from Nvidia, but this meant it tended to drive people towards Nvidia rather than steal them away.

[pjmlp]

Also apparently the reason behind Apple's cut with Khronos seems to be related to how OpenCL was managed by them.

[talldayo]

"Hey Khronos, can we tweak the OpenCL spec to be even more restrictive and higher-level, then rebrand it under our proprietary 'Metal' architecture so we can license it out to our competitors?"

"...no, but you could expand on OpenCL or Vulkan compute if you wanted. There are other spec stakeholders, we can't give you carte-blanche control, Apple."

"Why do you insist upon mismanaging the industry's APIs? Screw you guys!" <Beginning of mid 2010s "Khronos Drought" at Apple Computers>

[luyu_wu]

The obvious issue with both your points is that NVidia's competitors did do as such. AMD has had workable Linux drivers for many years now and there were numerous alternatives to CUDA pushed.

[AlexandrB]

A common talking point is that CUDA is a formidable moat for Nvidia, but - as someone who has never done AI dev - I'm curious to understand what makes CUDA so sticky. From an outsider perspective it looks like a re-run of DirectX vs. everything else but AI is not like gaming and end users often don't have to run the model themselves. So it seems like the network effects should be less than that for a graphics APIs.

[badsectoracula]

I don't know how it is nowadays but i remember trying CUDA back when GeForce GTX 280 was still a high end GPU. I didn't do anything fancy, i just tried to write a simple raytracer to get a feel of how it'd work.

The experience was incredibly simple: write C like usual but annotate a few C functions with some extra keywords and compile using a custom frontend/preprocessor/whatever-nvcc-was instead of gcc (i was on Linux - and BTW i heavily contest the notion that Nvidia drivers on Linux were "nightmare", they always worked just fine with both performance and features comparable to their Windows counterparts while ATi/AMD had buggy and broken drivers for years). Again, the experience was very simple, i even just copy/pasted a bunch of existing C code i had and it worked.

Later i tried to use OpenCL which was supposedly the open alternative. That one felt way more primitive and low level, like writing shaders without the shading bits.

In a way, as you wrote, it was kinda like DirectX: that is, CUDA was like using OpenGL 1.1 with its convenient and straightforward C API and OpenCL was like using DirectX 3 with its COM infested execute buffer nonsense.

After that i never really used CUDA (or OpenCL for that matter) but it gave me the impression that Nvidia did put way more effort on developer experience.

[disgruntledphd2]

Nvidia have invested a lot in CUDA, and they have C & Fortran bindings for a lot of scientific stuff, apart from all the DL/Gen AI stuff that's super hot right now.

Like, I started using CUDA (through frameworks) over ten years ago, and basically nobody has come up with anything competitive since then.

[kkielhofner]

> Nvidia have invested a lot in CUDA,

This is a significant understatement. For quite some time Jensen has been saying repeatedly that 30% of their R&D spend is on software. With the money-printing machine that is Nvidia if that holds they're going to continue to rocket ahead of competitors in terms of delivering actual solutions.

The "What are you talking about? AMD/Intel runs torch just fine!" crowd clearly haven't seen things like RIVA, Deepstream, Nemo, Triton Inference Server/NIM, etc. Meanwhile AMD (ROCm) still struggles with flash attention...

What these hardware-first (only?) companies like AMD don't seem to understand is that people buy solutions, not GPUs. It just so happens that GPUs are the best way to run these kinds of workloads but if you don't have a wholistic and exhaustive overall ecosystem you end up in single digit market share vs Nvidia at ~90%.

[mistrial9]

chicken and egg arguments.. good points and not untrue, but look elsewhere in this topic and see extensive anti-trust behavior, questionable license practices, deceptive public statements and deceptive handling of binary blobs. Very much like Intel - excellent tech in certain places, very mob-like business behavior in other places.

"What are you talking about? AMD/Intel runs torch just fine!" refers indirectly to the value of having competition in markets, not jump on the (well-funded,slick) monopoly bandwagon.

[pjmlp]

Tooling.

Since CUDA 3.0, NVidia has embraced a polyglot stack, with C, C++ and Fortran at the center, and PTX for anyone else.

Followed by changing CUDA memory model to map that of C++11.

Khronos never cared for Fortran, and only designed SPIR, when it became obvious they were too late to the party.

So not only has CUDA first level tooling for C, C++, Fortran, with IDE integration in Visual Studio and Eclipse, graphical GPU debugger with all the goodies of a modern debugger, it also welcomes any compiler toolchain that wants to target PTX.

Java, Haskell, .NET, Julia, Python JITs, .... there are plenty to chose from, without going through "compile to OpenCL C99" alternative.

Finally, the myriad of libraries to chose from.

CUDA is not only for AI, by the way.

[p_l]

The real moat of CUDA is that CUDA... works. Simply works out of the box, even on cheapest GPUs. Unless you want some specific high end stuff, everything will work on the cheapest GPU of given generation, with the same base tooling.

And because of that, their OpenCL implementation also works better than others. So there's more tooling not just from nvidia using it, because it. just. works.

Compare this with AMD, whose latest framework is a total mess of "will it work on this GPU?", sometimes needing custom wrangling to enable, etc. etc. and it's effectively supported only on the most expensive compute-only cards.

[deredede]

The difference is not just about APIs; CUDA has a single source file model that is dead easy to use whereas last I checked every competitor still had an outdated manual loading process that adds significant friction.

[zozbot234]

Doesn't SYCL also allow for a single-source-file model these days?

[deredede]

It is supposed to, yes. I was never able to set it up (admittedly I have not tried in a couple of years since I am not working with GPUs anymore) so I don't know how well it holds up.