[rwmj]

Communication overhead is the enemy of these highly parallel machines. The Connection Machines had many CPUs, but the unsung hero was the hypercube of connections between them [the clue is in the name]. According to Wikipedia it was a 12 dimension hypercube so every node had 12 high speed point-to-point I/O channels to adjacent nodes, which must have been a nightmare to implement and a nightmare to design software for. The cost of a CM-5 (Wikipedia again says $25 million) must have mostly been for this very specialised network.

It's hard to imagine this could have been competitive with a $25 million pile of beige PC boxes from the same era, but the PCs would have been starved of I/O (10 Mbps shared thick ethernet anyone?) so only applications which don't need much I/O between the nodes would be possible.

A "modern" CM-5 would ironically look much more like the pile of beige PCs, because it will have much less I/O -- these cheap chips only seem to have at most one or two fast channels (eg. ethernet and SDIO). There's no way to build these into a hypercube. It will be constantly limited by bandwidth and contention addressing other nodes in the cluster.

So I'd only build it for fun, not for practicality :-)

[stan_rogers]

SpiNNaker[0] is a species of that, with multi-dimensional connections in a toroidal surface configuration. It's ARM-based, largely because the chief developer/project head is Steve Furber. Along with interviews concerning the BBC Micro and ARM, Computerphile did a video with Furber concerning SPiNNaker[1].

[0] https://en.wikipedia.org/wiki/SpiNNaker [1] https://www.youtube.com/watch?v=2e06C-yUwlc

[rwmj]

Yes SpiNNaker looks very cool, also of course the BBC Micro connection as you say. I do wonder what the network architecture is, so now I'm going to have to watch that video you posted :-)

Edit: It's a toroid, which seems an unusual choice (because 2D) for something that's meant to simulate a brain. I wonder if a simple 3D cubic connection network would have been possible by adding more links between physically adjacent boards.

[generalizations]

I'm surprised it was worthwhile to build, since the whole thing has only 7TB of ram. I guess the total memory bandwidth is really good.

[generalizations]

I've noticed that a lot of those small arm chips are designed for external ram. I wonder if physically shared ram could overcome the I/O limitation.