[jonstewart]

As luck would have it, I'm going to write a POC next month with Spark Streaming and Flink. Should I also throw Storm into the mix? What do you think about Flink?

I generally can get by with Batch, but streaming makes a lot of our logic easier, and might allow us to handle some new situations. Flink looks pretty good on paper.

[cbsmith]

Flink looks really interesting to me. If it can get community momentum I might switch to it.

I'd definitely recommend including Storm for a POC. For any POC I'd warn that most of these streaming solutions are operationally a bit of a PITA, which tends to distort POC results.

[jonstewart]

We will be half-to-single rack scale for a while, so I'm not too too worried about operations--we also need to target solo workstations, which is the real challenge with any of these systems. If things go haywire, we can just restart processing later.

[cbsmith]

You might want to check out Kafka Streams.