As luck would have it, I'm going to write a POC next month with Spark Streaming and Flink. Should I also throw Storm into the mix? What do you think about Flink?
I generally can get by with Batch, but streaming makes a lot of our logic easier, and might allow us to handle some new situations. Flink looks pretty good on paper.
Flink looks really interesting to me. If it can get community momentum I might switch to it.
I'd definitely recommend including Storm for a POC. For any POC I'd warn that most of these streaming solutions are operationally a bit of a PITA, which tends to distort POC results.
We will be half-to-single rack scale for a while, so I'm not too too worried about operations--we also need to target solo workstations, which is the real challenge with any of these systems. If things go haywire, we can just restart processing later.
I generally can get by with Batch, but streaming makes a lot of our logic easier, and might allow us to handle some new situations. Flink looks pretty good on paper.