| Today's "exotic" (which is actually just high-end commodity) is tomorrow's middling. I'm not sure it's fair to summarize any one thing as "the entire point" of Hadoop, but, as I recall, it was, originally, an open source implementation of Google's Map-Reduce paper. Put another way, it was a way to bring Google's compute strategy to the masses. That said, the notion that there is "commodity, cheap, off the shelf PC hardware" and "exotic specifications" is completely a false dichotomy, especially in the face of what, for example, Google actually does. Google goes cheap. Very cheap. It's custom and exotic, just optimized for cost, but not absolute cost per nod, rather the ratio of cost for performance. That last part is what's missing from every single Hadoop installation I've personally seen (or that anyone I know has personally seen), the maximization of performance for cost. Instead, there's an inexplicable desire to increase node count by using cheaper nodes, no matter the performance. > Except that of course nowadays such hardware is just a couple of clicks away in AWS. I'm a bit unclear what the "except" means here. I don't believe AWS has the truly high-end specs available (and never has, historically, so we can reasonably assume it never will). It's also very not-cheap. |