Hacker News new | ask | show | jobs
by chicagobuss 3156 days ago
Absolutely, we used store-and-forward semantics when I used Kafka at a previous company to guarantee zero message loss (for data getting into kafka). Now that kafka streams provides exactly once semantics, you're in an even better spot to achieve this.
1 comments

You probably already know this, but obligatory reminder for others: Kafka's exactly-once delivery is not the same as exactly-once execution of an arbitrary workload. E1 mode is incredibly convenient, but it doesn't solve the tricky parts of, for example, hitting an external ACH to process a credit card payment. What if the external request times out? Fails silently? Fails loudly? Exactly-once delivery can help develop solutions in this area, but it doesn't eliminate this problem domain.
Of course, the key to any successful implementation of end-to-end exactly-once processing lies in the fact that you USE the kstreams APIs - including all semantics around "consumption is completed up to offset X"