Communication overhead is the enemy of these highly parallel machines. The Connection Machines had many CPUs, but the unsung hero was the hypercube of connections between them [the clue is in the name]. According to Wikipedia it was a 12 dimension hypercube so every node had 12 high speed point-to-point I/O channels to adjacent nodes, which must have been a nightmare to implement and a nightmare to design software for. The cost of a CM-5 (Wikipedia again says $25 million) must have mostly been for this very specialised network.
It's hard to imagine this could have been competitive with a $25 million pile of beige PC boxes from the same era, but the PCs would have been starved of I/O (10 Mbps shared thick ethernet anyone?) so only applications which don't need much I/O between the nodes would be possible.
A "modern" CM-5 would ironically look much more like the pile of beige PCs, because it will have much less I/O -- these cheap chips only seem to have at most one or two fast channels (eg. ethernet and SDIO). There's no way to build these into a hypercube. It will be constantly limited by bandwidth and contention addressing other nodes in the cluster.
So I'd only build it for fun, not for practicality :-)
SpiNNaker[0] is a species of that, with multi-dimensional connections in a toroidal surface configuration. It's ARM-based, largely because the chief developer/project head is Steve Furber. Along with interviews concerning the BBC Micro and ARM, Computerphile did a video with Furber concerning SPiNNaker[1].
Yes SpiNNaker looks very cool, also of course the BBC Micro connection as you say. I do wonder what the network architecture is, so now I'm going to have to watch that video you posted :-)
Edit: It's a toroid, which seems an unusual choice (because 2D) for something that's meant to simulate a brain. I wonder if a simple 3D cubic connection network would have been possible by adding more links between physically adjacent boards.
It's hard to imagine this could have been competitive with a $25 million pile of beige PC boxes from the same era, but the PCs would have been starved of I/O (10 Mbps shared thick ethernet anyone?) so only applications which don't need much I/O between the nodes would be possible.
A "modern" CM-5 would ironically look much more like the pile of beige PCs, because it will have much less I/O -- these cheap chips only seem to have at most one or two fast channels (eg. ethernet and SDIO). There's no way to build these into a hypercube. It will be constantly limited by bandwidth and contention addressing other nodes in the cluster.
So I'd only build it for fun, not for practicality :-)