[mlajszczak]

Idempotence can be very helpful when one strives for resiliency.

Suppose you have an application that processes tasks that (among others) call an external API. Both external API call and task processing can fail independently. Moreover, task processing can fail after API call succeeded. If you the external API is idempotent, you can simply retry on any task processing failure, no matter when it happened. It can simplify error handling a lot.

[NightMKoder]

One thing to note about this - idempotency is a very simple concept, but the implementation is actually quite hard. Case-in-point: the kafka producer API recently gained support for "idempotent/transactional capabilities": https://kafka.apache.org/documentation/#upgrade_11_exactly_o... . That is to say exactly-once persistence by using an idempotency key.

Everything looks bulletproof unless you take a step back. To connect with the example you gave: just having the kafka producer guarantee that one call to send() is idempotent is not enough. Your application needs to be able to be idempotent - e.g. if the same RPC/web request has to be retried on a different server. You need to be able to pass an idempotency key TO the kafka API - which you currently cannot do. The API currently allows for either a global(ish) lock or duplicated messages - so it's not quite idempotent.

Idempotency needs to be end-to-end, otherwise it doesn't work. Unfortunately that's very rarely the case - almost nobody tries to idempotently make XHR requests to their servers. In effect it's almost always easier to de-duplicate idempotently on read rather than attempt to write idempotently. It's a really hard simple problem with lots of corner cases.