I'm really hoping Modular.ai takes off. GPU programming seems like a nightmare, I'm not surprised they felt the need to build an entire new language to tackle that bog.
Yes, the problem isn't language, it is the entire stack. I think people focus too much on Mojo while ignoring the actual solution Modular has built, which is MAX.
The main idea here is that MAX provides a consistent API for both library authors (e.g vLLM, Ollama) to target, as well as for hardware vendors to integrate with - so similar to LLVM.
Basically, imagine if you can target Cuda, but you don't have to do too much for your inference to also work on other GPU Vendors e.g AMD, Intel, Apple. All with performance matching or surpassing what the hardware vendors themselves can come up with.
Mojo comes into the picture because you can program Max with it, create custom kernels that is JIT compiled to the right vendor code at rumtime.
Not sure I fully understand your comment, but I'm pretty sure the talk addresses exactly that.
The primitives and pre-coded kernels provided by CUDA (it solves for the most common scenarios first and foremost) is what's holding things back and in order to get those algorithms and data structures down to the hardware level you need something flexible that can talk directly to the hardware.
C, C++, Fortran, Python JIT from NVidia, plus Haskell, .NET, Java, Futuhark, Julia from third parties, and anything else that can bother to create a backend targeting PTX, NVVM IR, or now cuTile.
The pre-coded kernels help a lot, but you don't have to use them necessarly.
GPU programming isn't really that bad. I am a bit skeptical this is the way to solve it. The issue is that details do matter when you're writing stuff on the GPU. How much shared memory are you using? How is it scheduled? Is it better to inline or run multiple passes etc. Halide is the closest I think.
What are you skeptical of? I believe the problem this is solving is a framework that's not CUDA that allows low level access to the hardware, makes it easy to write kernels, and is not Nvidia only. If you watch the video you can write directly in asm if you need to. You have full control if you want it. But it provides primitives and higher level objects that handle common cases.
I'm a novice in the area, but Chris is well respected in this area and cares a lot of about performance.
The problem isn't the language, rather how to design the data structures and algorithms for GPUs.