[oaiey]

The bridge between inbox/outbox and queue is not perfect. But it derisks the process a lot. It is much saver to insert a (ideally idempotent) message into the database and then (without transaction) confirm it to the queue than running the whole business process. The likelihood the process will fail is much higher than the inbox / outbox. These patterns also keep your brokers queue empty and allows you to gracefully shutdown your systems.

[kokonuts]

At that point why not just keep the broker shutdown and have the parties read from the DB and update an ownership column? Personally, I find the whole message queue thing the typical waste of time, absurdly complex and pointlessly corrupting component you get when you pretend you are Twitter and have the problem of maybe delivering a lot of stuff single entry style instead of provable having handled a small amount of stuff that needs to be double entry handled because however small a professional gig is it still uses real accounting.

[mrkeen]

I somewhat agree with 'just use the db' over 'use the db as the state and the outbox'.

And message "queues" are probably a waste of time too.

Where you get a real benefit is in using a proper append-only ledger. This is a solved problem. Paxos and Raft both give you this on the theoretical side, and systems like Kafka give you practical implementations.

"Pull-based" systems are far, far easier to reason about. E.g., I'm going to update my packages now. I'm going to pull from git now. I'm going to GET news.ycombinator.com now. Imagine the opposite - news.ycombinator deciding to push the frontpage to whichever device I'm using, at the precise time I'm hoping to read it.

So pull is better, but if you can only pull, then how does anyone change any state? Push a new message into Kafka, and let it handle the switch from push to pull.

It may be absurdly complex, but it's the least absurdly complex option if you want to distribute. And if you don't want to distribute, you don't need outbox.

[oaiey]

I agree with you. For one, if you are small, build a good monolith and call it a day. So much more efficient than anything else.

Stream based systems where you maintain your own curser are a strong architectural decision similiar to a messaging systems. They also have their downsides.

Lastly on the last sentence: as soon as you need reliable processing of external input or output, inbox/outbox are needed. You are distributed because of your payment processor, because of your user email sending, etc. You do not want to block your core job processor just because the email server is overloaded right now.

[oaiey]

If you write a good monolith, you would do exactly that. You message between the subsystems and you are good with just adjust the "ownership" within a transaction.

If you are distributed you have the problem of shared databases. It would break your Microservice ownership etc if you all operate on one database. It is an anti pattern. For very good reasons.

In the past there have been distributed transactions between databases or other systems but they fell out of love due to their proprietary and limited nature (e.g. Microsofts MSDTC)