[withinboredom]

The best way to avoid distributed locks and transactions is to manually do the work. For example, instead of doing a distributed lock on two accounts when transferring funds, you might do this (which is the same as a distributed transaction, without the lock):

1. take money from account A

2. if failed, put money back into account A

3. put money into account B

4. if failed, put money back into account A

In other words, perform compensating actions instead of doing transactions.

This also requires that you have some kind of mechanism to handle an application crash between 2 and 3, but that is something else entirely. I've been working on this for a couple of years now and getting close to something really interesting ... but not quite there yet.

[candiddevmike]

> This also requires that you have some kind of mechanism to handle an application crash between 2 and 3, but that is something else entirely

Like a distributed transaction or lock. This is the entire problem space, your example above is very naive.

[withinboredom]

You _can_ use them, but even that won't save you. If you take a distributed lock, and crash, you still have the same issue. What I wrote is essentially a distributed transaction and what happens in a distributed transaction with "read uncommitted" isolation levels. A database that supports this handles all the potential failure cases for you. However, that doesn't magically make the errors disappear or may not be even fully/properly handled (e.g., a node running out of disk space in the middle of the transaction) by your code/database. It isn't naive, it is literally a pseudo-code of what you are proposing.

[nyrikki]

It is not a DTC, despite the DB world using ATM's wrongly as an example for decades, follow their model for actually moving money and you would be sent to jail.

"Accountants don't use erasers"

The ledger is the source of truth in accounting, if you use event streams as the source of truth you can gain the same advantages and disadvantages.

An event being past tense ONLY is a very critical part.

There are lots of way to address this all with their own tradeoffs and it is a case of the least worst option for a particuar context.

but over-reliance on ACID DBMSs and the false claim that ATMs use DTC really does hamper the ability to teach these concepts.

[singron]

The better version of this is sagas, which is a kind of a simplified distributed transaction. If you do this without actually using sagas, you can really mess this up.

E.g. you perform step 2, but fail to record it. When resuming from crash, you perform step 2 again. Now A has too much money in their account.

[withinboredom]

Sagas are great for this and should be used when able, IMHO. It's still possible to mess it up, as there are basically two guarantees you can make in a distributed system: at-least-once, and at-most-once. Thus, you will either need to accept the possibility of lost messages or be able to make your event consumers idempotent to provide an illusion of exactly-once.

Sagas require careful consideration to make sure you can provide one of these guarantees during a "commit" (the order in which you ACK a message, send resulting messages, and record your own state -- if necessary) as these operations are non-atomic. If you mess it up, you can end up providing the wrong level of guarantee by accident. For example:

1. fire resulting messages

2. store state (which includes ids of processed messages for idempotency)

3. ACK original event

In this case, you guarantee that you will always send results at-least-once if a crash happens between 1&2. Once we get past 2, we provide exactly-once semantics, but we can only guarantee at-least-once. If we change the order to:

1. store state

2. fire messages

3. ACK original message

We now only provide at-most-once semantics. In this case, if we crash between 1&2, when we resume, we will see that we've already handled the current message and not process it again, despite never having sent any result yet. We end up with at-most-once if we swap 1&3 as well.

So, yes, Sagas are great, but still pretty easy to mess up.

[hcarvalhoalves]

Here's how you can do this. You have 3 accounts: A, B and in-flight.

1. Debit A, Credit in-flight.

2. Credit B, Debit in-flight.

If 1. fails, nothing happened and everything is consistent.

If 2. fails, you know (because you have money left on in-flight), and you can retry later, or refund A.

This way at no point your total balance decreases, so everything is always consistent.

[withinboredom]

It can fail at your commas in 1&2, then you are just as broke as everyone else.

This isn't an easy-to-solve problem when it comes to distributed computing.

[hcarvalhoalves]

It should be an atomic transaction, double-entry style, so it can’t fail between commas.

The important thing is not having money go missing.

[withinboredom]

The only way that is possible is if the tables exist on the same database server. Otherwise, you are right back at distributed transaction problems.

[hcarvalhoalves]

That's the whole point of double-entry. You don't split the entries.

[withinboredom]

Then this is off-topic af. We are talking about distributed systems here.