[tbrownaw]

This only works for "embarrassingly parallel" tasks, where you have lots of completely independent things to do. Please remember that the whole world is not a web server.

This is not trading latency for thruput. This is spending hardware for thruput, with some overhead in latency.

The relevant generic performance number is CPU-seconds (for CPU-bound work), or I/O consumption (for disk-bound work), or in general how much of your bottleneck resource is consumed. Once you know your bottleneck, you can either improve your code to use less of that resource, or buy more of that resource.

As tasks become less embarrassingly parallel, throwing (non-serial) hardware at a problem increases communication overhead, and gives lower speedups.