| Full disclosure: I co-founded a startup (https://batch.sh) which focuses specifically on solving problems in the event driven space - we do event archive, search and replay. So this space is super interesting to me :) You are totally right - the CQRS pattern alone is not enough. In my experience, most shops that go down the CQRS path, also tend to make use of other event driven patterns like event sourcing. As with most things, there's pro's and con's and the most obvious con here is increased complexity in exchange for higher reliability and scale. Your advice is great, especially deciding on a structured message format such as protobuf. I would 100% avoid JSON schema as it's possible folks will forget to fill out fields. Protobuf has excellent support by now and there is a ton of support tooling for it. I'd be hard pressed to choose Avro at this point (unless you're a Java or Kafka shop that has it already in their ecosystem). In addition, if you are already on CQRS, I would advise looking into fully embracing event driven - by utilizing an event bus of some sort (rabbit, kafka, eventbridge), you could make message passing completely async (and avoid having to use gRPC, which is another layer of complexity, as you mentioned). A good friend of mine once said that in order to successfully do event driven arch, you must be OK with eventual consistency and I think that lies at the heart of this. If you are OK with eventual consistency, you understand the burden of complexity you're bringing on. |
MQTT is lightweight and fits nicely into IoT sources, backend streaming and even scalable frontend distribution to browser or mobile clients. I recently did a quick'n'dirty PoC with paho.js and D3.js for live charting in the browser.
In a pub-sub event-driven world, CQRS just means having distinct events for: Commands to do something; Query matches (CEP-style); and other Notifications that something has happened. Queries are compiled and unfolded into streaming operator graphs, delivering match events to the client. Queries and their operator graphs can be factored to share common sub-expressions (like the RETE rules algorithm).
A good start-up idea would be MQTT 'appstreamer' framework, with model-driven message formats, validation and APIs, together with pluggable engines for CEP queries, rules, workflows, constraint solvers and other business logic.