Hacker News new | ask | show | jobs
by ethangarofolo 2380 days ago
The technologies you listed are message brokers, while Message DB is a message store. The former transport messages, and the latter is a database specialized for storing message data and sourcing system state from those messages. Kafka can, for example, move a lot of events around, but it isn't suitable for event sourcing for 2 key reasons.

The first is that one generally has a separate stream for each entity in an event-sourced system. Streams are sort of like topics in Kafka, but it would be quite challenging to, say, make a topic per user in Kafka. The second is Kafka's lack of optimistic concurrency support (see https://issues.apache.org/jira/browse/KAFKA-2260). The decision to not support expected offsets makes perfect sense for what Kafka is, but it does make it unsuitable for event sourcing.

Being built on top of Postgres, Message DB gives you access to event sourcing semantics using a familiar database technology.

We use Message DB in our production systems, and I'd be happy to talk more about it if you have other questions. We've found it very reliable.

As a disclaimer, I'm listed as a contributor at the Eventide Project, the project Message DB was extracted from, though I did not write any of the code behind Message DB.

3 comments

I think the biggest difference is that it’s a Message Store as highlighted by the preceding comment.

Some clarity by example: In financial systems, it’s extremely important to keep track of all the transactions between your microservices (if your architecture is based on that). You could potentially lose a message delivered to you via a broker if your service fails to write it to a persistent storage. If the producer of that message never stored that message or it was produced on wire and transmitted, there is no way to recover it anymore. A system designed around a Message Store can mitigate such problems. You can build that similar architecture with brokers as well but for each of your application, you will have to implement something analogous to a Message Store to handle idempotency and things like that.

Incorrect. Kafka is a message _storage_. The server can be seen as a distributed persistent append-only log. All broker logic is encoded in the client. Source: maintainer of one of client implementations.
Is this using NOTIFY/LISTEN to stream messages or some other way to get new messages as they arrive?
Good question. Consumers poll for updates. One of the stored functions in Message DB is designed for this very query.

Polling sounds very crude, but for the systems Message DB is designed for, it's a virtue. No back pressure problems, for example.