[seanmcdirmid]

> yeah, I always map CUDA blocks to racks and CUDA threads to blades ;)

Sounds like Jeff Dean.

> Nobody said it should be MPI, but there are still many hybrid architectures with MPI handling distribution and actual compute done using OpenMP or accelerators.

MPI or even RPC works fine as a control mechanism, just not as a critical performance-sensitive abstraction, where we care about the width and speed of the pipe, and MPI is nothing like a pipe!

> In my system I use Erlang/OTP to handle distribution and concurrency and OpenCL for data-parallel compute.

This is quite reasonable. Once one understands the difference between concurrency and parallelism, they can pick appropriate tools to deal with each. As long as they confuse the issues, they'll make bad choices.