[j-pb]

It depends heavily on the kind of work. If you have a large scale simulation that needs to be partitioned like a weather system you are IO bound and need as thick interconnects as possible. However there are some problems which are very hard computationally but not very large. Basically everything in NP and exp is a good candidate. There you can distribute the same problem to a bazillion systems with a different starting configuration and let them run until one of them obtains a solution.

If you look at the BOINC project those are basically all problems of this kind. Folding proteins like folding@home does for example. The description of a protein is fairly small, a couple megabyte max. However it takes a long time to simulate the behaviour, since chemistry is a messy probabilistic process with lots of back and forth. Nature does this on trillions of proteins at the same time within nanoseconds, and while we cannot reasonably increase the simulation speed of an individual protein, we can at least simulate as many proteins at once as possible.