[NightMKoder]

One thing to note about this - idempotency is a very simple concept, but the implementation is actually quite hard. Case-in-point: the kafka producer API recently gained support for "idempotent/transactional capabilities": https://kafka.apache.org/documentation/#upgrade_11_exactly_o... . That is to say exactly-once persistence by using an idempotency key.

Everything looks bulletproof unless you take a step back. To connect with the example you gave: just having the kafka producer guarantee that one call to send() is idempotent is not enough. Your application needs to be able to be idempotent - e.g. if the same RPC/web request has to be retried on a different server. You need to be able to pass an idempotency key TO the kafka API - which you currently cannot do. The API currently allows for either a global(ish) lock or duplicated messages - so it's not quite idempotent.

Idempotency needs to be end-to-end, otherwise it doesn't work. Unfortunately that's very rarely the case - almost nobody tries to idempotently make XHR requests to their servers. In effect it's almost always easier to de-duplicate idempotently on read rather than attempt to write idempotently. It's a really hard simple problem with lots of corner cases.