[paulsutter]

Disk io has negligible CPU overhead. Preempted for a millisecond? A millisecond is millions of instructions. You're off by orders of magnitude. No matter where the disk is located, the disk io has to go across the network. If network capacity truly is the bottleneck, you have a different design problem and you cant exploit the CPUs.

EDIT: I still dont buy it, but I will give some thought to the synchronous/lockstep nature of the environment.

[rbabich]

That was a straw man (sorry). There's also the overhead of maintaining consistency, synchronizing metadata, etc. I don't think assuming 0.1% CPU overhead for Lustre is a terrible estimate, but even if it were much lower, the argument would still hold (at least at the scale of Titan).