| > So you need a fat pipe and some middle man [...] waiting subscribed to topics and waiting for messages. Your use case is not Mr everybody use case nor the one presented in the article. Most usages of Kafka I have encountered in the wild is for notification delivery or telemetry report and of the order of few ~1000 msgs/s. You do not need a fully distributed ordered log system for that. MQTT does the job for a fraction of the complexity and operational cost. > . Say that you can do C1M happily, you still need 20 brokers to serve egress for all your connected clients. Now imagine that you have 100 brokers, Even at these scales, you can find some commercial MQTT brokers going over 20M msg/sec nowadays. With OSS solutions, you could get your way there with some HAProxy + your favorite MQTT broker behind DNS load. balancing as long as you do not requires HA, scale only should not be the issue. It would even play pretty nicely with anycast if you want to place your brokers at edge close to your customers and do some proper partitioning. That is currently pretty much the case presented in the article. They just advertise telemetry report (very likely not HA) injected by a time series database. Once again, if what you need is fully ordered distributed commit log for a complex scenario of event sourcing: Go for it, go for Kafka, it has been designed for that.
But it is just not the case of most Kafka instances I see deployed in the wild, these ones are generally the result of quick Google-search driven engineering. |