[timlatim]

GPUs do not have thousands of physical cores, though. What's called a "CUDA core" is essentially a programming slash marketing abstraction for a SIMD lane. A closer analogy to a CPU core would be a Streaming Multiprocessor (RTX 4090, for instance, has 128 of them). But that comparison is still moot, because GPUs are simply not designed for executing branchy scalar code. They'd be laughably slow at it, so outside of easily vectorizable code that already takes advantage of SIMD instructions, I don't see how you could offload any CPU tasks to a GPU.

[blagie]

I think the major architectural leap would be MIMD with a standard ISA.

I think something along the lines of a Pentium Pro or an ARM core. Pentium Pro had 5.5 million transistors. A modern CPU has about 1000x more, so about a thousand Pentium Pro-grade processors would fit in die like a modern 7770X.

I'd take that over my GPU any day.

The hard and expensive part is, obviously, memory, cache, and interconnect. The even harder part is software. And the less hard part I'm intentionally oversimplifying is power consumption.

[lizknope]

https://en.wikipedia.org/wiki/Multiple_instruction,_multiple...

An example of MIMD system is Intel Xeon Phi, descended from Larrabee microarchitecture.

https://en.wikipedia.org/wiki/Larrabee_(microarchitecture)

Its x86 cores were based on the much simpler P54C Pentium

https://en.wikipedia.org/wiki/Xeon_Phi

That is the core from the generation before the Pentium Pro. Larrabee was supposed to be a GPU, wasn't good enough to compete at that, then they rebranded it as Xeon Phi but cancelled it a few years ago.

[jacquesm]

I tried buying those but they wouldn't let me.

[monocasa]

Well, FWIW xeon phis are about $40 on ebay now. Theyre obviously not much more than toy's now.

[jacquesm]

I'd love to see solutions like that. But what you are talking about is probably a niche market.

[blagie]

I don't think so. I think the core problem are network effects and momentum.

If someone could wave a magic wand, and there were OS, app, compiler, video game, etc. support for both MIMD and current architectures, I think MIMD would take over overnight.

Most of what computers do is ridiculously parallel. From each browser tab getting an isolated CPU, to having a spreadsheet spread out among cores, to rendering fonts in a document.

However, given a universe with trillions of dollars invested in the status quo, a disruption would need some sort of rather complex pathway, with some niche markets, some growth strategy, etc. As someone pointed out, Intel tried with Phi and failed.

[jacquesm]

I think the big driver could be security.