| > There is no value in returning a "hey this is in progress" error code to a client when transactions are short Fair enough. So basically your approach is to wait until the first request completes to decide how to respond to the second request that came in with the same idempotency key. However, that would seem to me to imply that when the second request comes in, you check its idempotency key, realize you've already received a request with that key and you're processing it, and don't do anything else with the second request until the first one is completed. In particular, you don't have the second request trigger the start of another transaction. But elsewhere in this thread, you've said you would start a second transaction based on the second request, and let your database's transaction mechanism tell you that it's a duplicate when you try to commit it. Why would you do that if you've checked the second request's idempotency key and you know it's a duplicate? > You seem very focused on long-running orchestration type systems. I'm not focused on anything except getting what I thought would be a simple answer to a simple question. The above seems to provide that (though it still leaves a question open, as above). That's all I wanted. > You don't necessarily share the idempotency key between the "start process" request and the "check status" request. I'm not talking about a "check status" request. The scenario I've been asking about all along is when a second "start process" request comes in with the same idempotency key as a previous "start process" request, while the process is still in progress. |
Part of the problem here is that we're confusing how do you structure the API (replay? 409? something else?) with how we implement the API. The original article (and my original response) focused on API structure. We're wandering into the details of implementation, which is fine, but there are of course many ways to do the implementation. Some simpler than others.
Here's the simplest and most reliable way to implement idempotency for a trivial "create payment" operation, where the client submits an idempotency key. This pattern is incredibly common. Every request looks something like this:
* Start a transaction
* Lookup "does this idempotency key already exist"
* If it doesn't, insert the payment record with the idempotency key
* Commit the transaction
* Return the result. Successful insert is always 200OK. "key already exists" results in either replay of the original result (Stripe model) or an explicit error like 409 (my favored approach, still ubiquitous in ecommerce, and very common in financial APIs that predate Stripe).
Does that help? If you're using your database to handle concurrency, you need every request to start inside the transaction. You can't check the idempotency key outside of the transaction or you can't guarantee once-and-only-once behavior.
[Before someone mentions it, yes you can use a unique constraint instead of an explicit transaction, and this is conceptually identical - the check-for-dup transaction is inside a single INSERT]