[winwang]

I'm not sure about the rest of your comment, but we would likely still want GPUs even with highly multicore CPUs. Case in point: the upper-range Threadripper series.

It makes sense to have two specialized systems: a low-latency system, and a high-throughput system, as it's a real tradeoff. Most people/apps need low-latency.

As for throughput and efficiency... turns out that shaving off lots of circuitry allows you to power less circuitry! GPUs have a lot of sharing going on and not a lot of "smarts". That doesn't even touch on their integrated throughput optimized DRAM (VRAM/HBM). So... not quite. We'd still be gaming on GPUs :)

[zackmorris]

Ya I winced after I wrote that and am considering taking a break from posting for a while. I'm just really tired of suffering the status quo because many people love it if it aligns with their worldview, while people like me who see things a little differently seem condemned to struggle in isolation. I've always admired Tesla, not Edison, but the Edison way of doing business seems to win out and suppress the people on the fringe.

As for GPUs, they are useful for languages like GNU Octave (MATLAB) and Julia. Although there's a code smell with that. The matrix languages that would benefit most from SIMD seem to be the ones that aren't hardware accelerated on GPUs. Something has gone wrong:

https://news.ycombinator.com/item?id=8302256

That was over 10 years ago and I doubt the situation is any better today. Misaligned incentives.

I feel that the missing link is direct hardware access to the GPU ALUs. Without being able to transpile CPU code to GPU code and vice versa to provide bare metal multithreading and vector operations, we miss out on a whole branch of computer science. We can't do the interesting experiments that I always rant about, which would make stuff like genetic algorithms borderline trivial. We're stuck with cookie cutter solutions like OpenGL, Vulkan, Metal, etc.

Yet we still praise Nvidia, even though like Microsoft/Intel in the 1990s, they're probably most responsibly for stifling innovation in multicore computing.

For what it's worth, you're right about the latency/throughput tradeoff. I don't mind stuff like Intel's integrated graphics or Apple's M line of processors with integrated GPU and NPU cores for raw throughput and efficiency. I just think they're silly because they operate at the wrong layer: 1) CPU 2) multicore 3) GPU. There's no layer 2. Why is that?

[winwang]

Gotcha, no problem.

> while people like me who see things a little differently seem condemned to struggle in isolation

Haha, you should consider founder life then. I think many founders feel like this.

The issue with transpiling to GPUs is that it is really hard to do that in a sensible, performant way. Something something a sufficiently smart compiler... There is plenty of @jit in Python, and there's TornadoVM for JVM.

Nvidia's CUDA has been around since the 2000s and is somehow still best way to program for GPUs still. Probably the best SIMT-oriented stack at all, even. From my perspective, it's everyone else stifling themselves.

[zackmorris]

I realized after responding that I forgot what I meant to add - that I shouldn't have brought neurodivergence into the conversation. I believe that proponents of the status quo are vilifying outside-the-box thinking and the innate sense of justice prevalent in people on the spectrum because they feel threatened by them. But that's a generalization that makes assumptions about people and isn't constructive, so I'm sorry for saying that.

Anyway, ya I can see how founders get pretty well ostracized by the business community. I tried to be one back when I thought that shareware mailbox money meant winning the internet lottery. Statistically, it's highly unlikely that anyone can make it on their own now. It's not just that the system is rigged, but that founders who exit often take the money and run rather than changing the systems of control that they managed to overcome. To me, rising wealth inequality and deliberate ignorance of survivor bias indicate that the situation is getting worse, not better.

You're right about the sufficiently smart compiler, I hadn't heard that term in a while. I believe that functional programming (FP) and imperative programming (IP) are at opposite ends of the spectrum, and that FP is about simple formal determinism while IP is about easy informal convenience. I also believe that IP can never be fully optimized, due to the halting problem, exponential complexity from mutability, etc. Whereas FP might be optimizable because it can be statically analyzed. Usually IP code is broken down into FP segments which can be optimized, then maybe mutable code gets approximated with impurity or monads or left as a bottleneck. I dream of writing an IP language where all variables are const and there are no pointers or references, that's performant by way of copy-on-write and other techniques where the runtime does the heavy lifting instead of humans, so that it can be converted to/from FP and fully optimized. Without that, I don't see a practical way to convert CPU code to GPU code.

So it's really 2 problems that are unlikely to be solved any time soon:

1) we're using the wrong hardware

2) we're using the wrong software

The status quo won't fix this, and the game is rigged to prevent independents from fixing this, and that's why I'm not seeing any movement on these problems after observing them for over a quarter century.

AI will likely make things worse by glossing over the real work of fixing the fundamentals. Rather than writing provably correct code in an FP language like lisp and verifying it with behavior-driven development (BDD) integration tests for example, we'll trust the IP code written by AIs and skip verifying it.

So it's been over for a long time, and getting to be even more over than it was. Defeatism is the safest bet now, which I find heartbreaking. And that's what people will hear when they read my words, even though I'm fighting for a revolution in how we write code and how fast it runs. That's why it's over.

Edit: it's never over :)