Hacker News new | ask | show | jobs
by Fishkins 1822 days ago
Say you have a batch job that performs tasks A–E, and each of those tasks might process thousands of records. At some point, something will go wrong that causes the process to crash, hang, or error over many records. If the code is not idempotent, you need to investigate exactly where things started going wrong and figure out how to resume the process at that point. You don't want to reprocess records and e.g. send out duplicate emails or double-increment some number you're tracking. If the code is idempotent, conversely, you can just start the whole process over again without having to worry about any of that.

Similarly, many systems involve consuming from some message queue. It's basically impossible to guarantee exactly-once delivery in most systems. You either have to risk missing a message, or having it delivered multiple times. If you're running idempotent code, you can always err on the side of redundant delivery without any ill effects.

2 comments

Last week I ordered a screen protector for my phone. I got two boxes in the mail. I thought I ordered twice by mistake but they had the same order number on the packing slip.

My immediate thought was that some order processing step somewhere is not idempotent.

How do you make something idempotent if one of the effects is sending an email?
You can't make everything idempotent.

You should still try and make most things idempotent.

Side effects go to the border of the app. You keep track if for some message X you have sended the email, so you garantie that you send it once.