[jahitr]

Storm is pretty much dead. Spark, from my point of view, is the current forerunner.

It support mostly all use cases. Batch jobs. Streaming jobs. Work is being done to reduce the batch time to ms.

Comercial support is very good from the guys at Cloudera and Databricks.

My 2 cents. I hope It helps someone.

[pcx]

I love Spark, it has great documentations and Scala! But how is Storm dead?! The documentation is not good enough yet, but there is a strong userbase and it is being actively used in a lot of companies.

[cbsmith]

Storm is anything but dead. Spark streaming is definitely not all there yet.

[jonstewart]

As luck would have it, I'm going to write a POC next month with Spark Streaming and Flink. Should I also throw Storm into the mix? What do you think about Flink?

I generally can get by with Batch, but streaming makes a lot of our logic easier, and might allow us to handle some new situations. Flink looks pretty good on paper.

[cbsmith]

Flink looks really interesting to me. If it can get community momentum I might switch to it.

I'd definitely recommend including Storm for a POC. For any POC I'd warn that most of these streaming solutions are operationally a bit of a PITA, which tends to distort POC results.

[jonstewart]

We will be half-to-single rack scale for a while, so I'm not too too worried about operations--we also need to target solo workstations, which is the real challenge with any of these systems. If things go haywire, we can just restart processing later.

[cbsmith]

You might want to check out Kafka Streams.