Hacker News new | ask | show | jobs
by streetcat1 2325 days ago
No. Because the number of use cases where you have 1TB or 2TB of data is smaller in comparison.

Also, the rest of the use cases (which fits into a single machine memory now), can be handled much more efficiently with memory base algorithm, instead of I/O based algorithms.

The goal of Hadoop, as well as most of the theory on disk-based indices (E.g. BTREE), was to overcome the I/O bottlenecks. But as memory is getting bigger and cheaper there is a trend to drop Hadoop in favor of reading data directly from the cloud and into memory.