[bitwize]

Building the communication fabric it takes to make those oodles of identical processors to exchange and share data quickly so they don't get bogged down in their own communication overhead is a profoundly interesting problem, and by "profoundly interesting" I mean "call Richard Feynman in to help you solve it":

https://longnow.org/essays/richard-feynman-connection-machin...

Besides which, at that level the goal is not to go "look at this cool thing we built", it's more like "how do we cheaply and effectively build something that can solve these massive weather/nuclear explosion/human brain/etc. simulation problems we have?" and if ganging together lots of off-the-shelf CPUs/GPUs achieves that goal with less time, effort, and cost than building super-custom, boutique-schmoutique hardware, so be it.