[mihaic]

The more powerful processors become, the less I feel there's a need to build supercomputers.

Thinking about it, the most powerful supercomputer in the world is pretty much a million consumer processors, working in parallel. That's going to stay pretty constant, since cost scales roughly linearly.

If X is the processing power of $1k of consumer hardware, the bigger X gets, the less there is a difference in the class of problems that you can solve with X or X * 1e6 processing power.

[uniqueuid]

Sure, but consumer hardware does not have infiniband or other high-bandwidth interconnects. That means you can have at most ~1-2TB of ram accessible at any point. Some problems need coordination, and when you're back at OpenMP etc., a supercomputer suddenly makes sense.

[mihaic]

I agree right now, I'm thinking maybe in 15 years you can have >1PB on a single machine, and then those problems that don't fit in that space but that fit in a supercomputer become fewer. 2050 will be within out lifetime.

Basically I'm estimating the benefit ratio to be (log SupercomputerSize - log ConsumerSize)/log ConsumerSize, and that keeps decreasing.

[uniqueuid]

You're not wrong.

The set of problems that fit into a single node is growing. At least in some fields where the added benefit of more data is less important than, say, more precise measurements.

[mastax]

The coherent memory interconnects between nodes is typically what makes supercomputers different than just a bunch of consumer hardware. It allows different types of programming or at least makes them easier.

[jabl]

It's a very fast, very low latency network fabric. But it's not coherent in the sense of cache coherent multiprocessors, and it doesn't offer shared memory style programming where you'd just load/store to addresses that happen to be mapped to another compute node somewhere in the system.

[l33t2328]

I thought DMI allowed for exactly those kinds of load/store operations

[hdjjhhvvhga]

By the way, while cost may scale linearly, the number of cores doesn't[0]. We have more powerful computers in our pockets than Cray supercomputers from the 80s. And I feel we still haven't learned how to use these cores in an efficient way.

[0] https://i.imgur.com/Gad4cKk.png

[xhkkffbf]

If you think of it this way, aren't some botnets truly the most powerful computing systems?