[eiopa]

I would've loved to hear more details about why you built your own. For example, you mention that Elastic Search wasn't deployed at your scale and there was some talk about machine footprint, but it doesn't explain how your solution compares to something like ES.

Did ES just didn't scale when you tried it? Is your solution better/faster? If so, by how much and on what workloads?

Contrast this with something like RocksDB. They just show you the numbers - http://rocksdb.org/

[tracker1]

I think it depends partly on implementation and partly on their use case... The way ES works probably doesn't do well past a couple hundred nodes, max and even then likely has some real issues. Though they could have several independent clusters and shard out their users. What it looks like they implemented will scale better (to thousands of servers), and probably work better with their design.

That said, I'm somewhat surprised they didn't just try doing custom index/word counters against a larger cassandra cluster, which would scale well while still being somewhat out of the box as a software approach. I didn't thoroughly read through the article, but not sure of their use of stemming/mapping for word bases either.