|
> It works fine on the happy path, but if mysql goes down, you lose things. Why do you have to lose things if the DB goes down? Agreed, untuned & unconfigured MySQL (and MongoDB) out-of-the-box can lose things due to bugs and design issues, but that is the case even when they are running. However, DBs, in general are made precisely for the purpose of safely storing things and not losing them. OTOH, the number of Kafka setups I have seen that'd lose things when something goes down ... maybe this is not a guaranteed win for the Kafka side of arguments. > Having a distributed, resilient queue has availability benefits. High availability is not a function exclusive to Kafka. On the other hand, there's some functions that may come in handy to use in a queue that Kafka simply cannot provide, but DBs can. Off the top of my head: ACID, instant scalability (both up and down) of consumer groups, and the sheer flexibility (and power) that comes with a DB in general. ---- Overall, there's some merits to using a distributed log as a message queue, sure, but there are also merits to using a DB for that. |
Was the data actually acked across all replicas by Kafka? If so, very interested in how it was lost.