[lenjaffe]

The main problem with running 10,000 cores is partitioning your program so that inter-core communication can happen effectively.

You could have several very large thread pools, each performing a different task in a processing pipeline.

You could set up a big SIMD system. Bu didn't the Connection Machine already do this, to the tune of 64k cores?

Maybe you're trying to figure out how to apply the model in a large distributed system where "core" is actually some cloud compute instance. You're back to devising effective ways to communicate with all your cores.

In the end, massively parallel processing, beyond generalized horizontal scaling tends to be designed for a particular problem domain. Some problems lend them selves to vectorization, others to SIMD decomposition, others shared-nothing multiprocessing with message passing between cores.