[throwaway_pdp09]

http://www.frankmcsherry.org/graph/scalability/cost/2015/01/...

"Rather than making your computation go faster, the systems introduce substantial overheads which can require large compute clusters just to bring under control.

In many cases, you’d be better off running the same computation on your laptop."

My limited experience fits this in that a bit of smarts on a single box beats a bunch of boxes.

(the link is a very good read BTW)

[yencabulator]

The biggest problem with that write-up is ignoring availability. "Fast all the way up to the crash" can be much worse than slow and steady.

Of course, for a batch job with a runtime under 11 minutes, that probably doesn't really matter too much. Just don't generalize that too much.