Hacker News new | ask | show | jobs
by bushbaba 1652 days ago
Shopify uses GCP. Curious why flink over beam/dataflow?
1 comments

Great question. We actually tried Dataflow before pivoting completely to Flink. Dataflow has many benefits, and I had a great experience for batch use cases at a previous company, but for stateful streaming the team identified limitations. On top of that Flink is open sourced, which is a huge benefit for us because we can more easily debug deeper issues and contribute to its direction. That said, it is more work to use Flink compared to Dataflow out of the box —- our platform team is looking to reduce this friction and hopefully contribute to Flink in the process.