Hacker News new | ask | show | jobs
by themoop 1581 days ago
This is where you might want to use the outbox pattern. If you depend on dual writes you will definitly have consistency issues at some point
2 comments

The question at hand will always reduce down to the "Two Generals problem" [0]. The outbox pattern is a nice separation of concerns and gives at least once consistency as long as the forwarding happen before writing the event as processed in the outbox. Both side effects happening atomically through all failure cases is impossible.

To solve that issue you need a cooperating destination for your writes which handles de-duplication. Then you get into the weeds of causality, ordering and idempotence. Ugh.

For some real world examples see the AWS Kinesis documentation which says that any application using Kinesis must be able to handle duplicate records [1].

> There are two primary reasons why records may be delivered more than one time to your Amazon Kinesis Data Streams application: producer retries and consumer retries. Your application must anticipate and appropriately handle processing individual records multiple times.

[0]: https://en.wikipedia.org/wiki/Two_Generals%27_Problem

[1]: https://docs.aws.amazon.com/streams/latest/dev/kinesis-recor...

For anyone who's interested in the outbox pattern, I also blogged about that: https://blog.frankdejonge.nl/reliable-event-dispatching-usin...