| Clickhouse is much more resource efficient in many cases but is less flexible and importantly extensible than Druid. Druid can easily be extended through available 3rd party extensions and you can write your own to implement custom serialisation formats, aggregations, connect to new streaming systems, read directly from whatever cold storage you have etc. In the Clickhouse model you have to work out a lot more of that stuff yourself though these days it can read from Kafka directly which is useful. Some things that are important for Clickhouse vs Druid at big scale is the rather large difference in indexing approaches.
Clickhouse uses bloom filters and other probabilistic data structures to index large chunks of data, for the most part though actually checking for rows requires a full scan of that chunk to strip false positives. This is different to Druid which uses full inverted indices for dimension filtering. The tradeoff is basically Clickhouse is cheaper, especially when scaling out but Druid is faster especially when the cluster is under heavy concurrent query load, like serving analytics dashboards or data exploration interfaces to users. Clickhouse excels when you want to scan most but not all the data most of the time. Namely reporting or bulk analytics queries that will hit most rows in a block. I consider both to be excellent databases. |
One thing I wanted to add with regard to performance. Druid does indeed get a big boost from the fact that it uses inverted indexes for filtering. It also gets a boost from having a wide variety of approximate algorithms you can use if you want (for things like topN, count distinct, set difference/intersection, quantiles, etc). But straight scan performance has been improving quite a bit recently too.
The biggest change related to straight scan perf is fully vectorizing the query engine, which is partially done as of the latest release (0.17): https://druid.apache.org/docs/latest/querying/query-context..... In benchmarks, the implementation so far has been posting row scan rate improvements in the 2-3x range. I expect we'll be able to round it out and have it work for all queries over the next couple of releases. The multiples involved mean this is quite meaningful if you do a lot of straight scans.
There's plenty of other stuff going on too: our latest release added parallel merging of large result sets. Our next one (0.18) is going to add a new, more efficient hash aggregation engine. That next release is also going to add a JOIN operator -- not perf related, but probably the number one most requested feature.