[BenoitEssiambre]

Asynchronous queues make your data out of sync (hence the name) and inconsistent one of the main downsides of microservices. Their use should be minimized to cases where they are really necessary. A functional transactional layer like postgres is the solution to make your state of truth accessed in a synchronized, atomic, consistent way.

[tombert]

No, I disagree with that completely actually.

Functions and handlers should not care where data comes from, just that they have data, and a queue is the abstraction of that very idea. Yes, you lose atomicity but atomicity is generally slow and more problematic has a high amount of coupling.

I don’t agree that being out of sync is the main downside of microservices; the main downside is that anything hitting the network is terrible. Latency is high, computers crash, you have to pay a cost of serialization and deserialization, libraries can be inconsistent, and zombie processes that screw up queues. Having stuff in-process being non-synchronized wouldn’t even hit my top five.

ETA:

I should be clear; obviously there are times where you want or need synchronization, and in those cases you should use some kind of synchronization mechanism, like a mutex (or mutex-backed store e.g. ConcurrentHashMap) for in-process stuff or a SQL DB for distributed stuff, but I fundamentally disagree with the idea that this should be the default, and if you design your application around the idea of data flow, then explicit synchronization is the exception.

[BenoitEssiambre]

I'll agree that the network layer adds more problems to microservices, but even with a perfect network, they are problematic. Everything being out of sync, (if they are stateful microservices which queues imply), is one big issue. Things being interconnected in broad global scopes instead of more locally scoped is the other big issue.

The more you have globally interconnected and out of sync states, the less predictable your system is.

The solution is to be as hierarchical, as tightly scoped, as functional and as transactional as you can.

That's how you tackle complexity and create intelligent systems: https://benoitessiambre.com/entropy.html

[tombert]

I think we are at a fundamental disagreement on this.

You can make asynchronous code predictable if you utilize something like TLA+, or treat the code as a protocol system.