Hacker News new | ask | show | jobs
by nl 2899 days ago
You seem to have missed everything the comment you are replying to says.

There is no dispute with the size or amount of compute available on desktops.

1 comments

No idea what cost that comment refers too. If you are Facebook or YouTube or Twitter or Amazon sure you need your five thousand man PAID army of content reviewers to keep the data clean cause the data is mostly useless. But if you are running Wikipedia search or Stackoverflow's search or the team at the library of congress please go and take a look at the size of these teams and their budgets and their growth rates.
Are you conflating "single server" and "desktop computer" here? It sounds a lot like you might be?

Because almost everyone already uses the model of co-locating the search index and query code on a single computer (both Wikipedia and Stackoverflow use Elastic Search which does this).

They use multiple physical servers because of the number of simultaneous requests they serve.

This has never been the use-case for Hadoop.

I've built Hadoop based infrastructure for redundantly storing multiple PB of unstructured data with Spark on top for analysis. This is completely different to search.

That's very different to the Wall St analyst running desktop analysis in Matlab, or the oil/gas exploration team doing the same thing.