[scott_w]

Not in the payments world. If you’re 99% done but only the bookkeeping failed, then it’s likely that money is changing hands and you need to deal with that fact. Payments are not an atomic infrastructure and you cannot magic that into reality.

[stickfigure]

Payments are multistep but each of the steps needs to be atomic. The "create payment" operation must be transactional and the communication channel between you and the processor must be idempotent so you don't inadvertently create multiple payments.

The fact that payments have a settlement process is not relevant to this discussion.

[scott_w]

> The "create payment" operation must be transactional and the communication channel between you and the processor must be idempotent so you don't inadvertently create multiple payments.

Yes, I agree. You want to generate a token, persist it locally and use that to communicate with the payment gateway, so re-submissions use the same key and either error or return the transaction state.

> The fact that payments have a settlement process is not relevant to this discussion.

I wasn't talking about settlement, I was talking about the processing aspect. What I meant was: once you kickstart the process with the gateway, money is highly likely to change hands as a result. This means a process of:

1. POST /checkout

2. Create token

3. POST to payment gateway with token

4. Wait for gateway to return

5. Persist transaction/error

6. Return success/error

What is needed is to persist and return the token to the caller before contacting the payment gateway, to make a check + retry mechanism possible.

And yes, I've seen code that follows steps 1-6 exactly as I've described and, yes, all the problems you imagine would occur from those steps have occurred at one time or another.

[stickfigure]

This is one possible API but it's not the dominant way payments are exposed by payments providers. Stripe, Amazon, Paypal, et al do this differently. They're fine.

[scott_w]

I pointed to Braintree in another comment where this is very much not fine. Also these providers operate at a lower level of the stack than we do, so they have finer control over the process than you or I.

Even then, it’s not fine because those requests might time out, or your request times out waiting for theirs. Just because your provider abstracts behind one API doesn’t mean you necessarily can!

[stickfigure]

This problem is the entire point of the idempotency key system. You (the client) must retry the payment until success (where APIs that provide "this idempotency key already seen" is considered success). The idempotency key prevents you from creating duplicate payments.

500 errors, network timeouts, etc all happen. We can't run 2PC transactions with Stripe, so you need durable retries. People run billions of dollars through these APIs every day. It's fine.

[scott_w]

I’m not disagreeing with that point: I’m just pointing out that the idempotency key must be persisted in your own system before the payment is submitted. Some gateways (Braintree) will let you submit no key and just give you one on request completion.

My point is that you can’t rely on this specific mechanism because request failure does not mean the payment is not going through!

I’m not sure where we disagree so I must ask: do you disagree with what I’ve written and, if so, what and why?

[jmalicki]

And if there is part of the process that isn't idempotent, you want to make that surface area as small as possible, so that only failures at that one discrete step can cause issues.