| Damn, I got lazy on my reply & just hoped nobody went further, but well played on digging deeper. 5. Kafka is silly expensive Pulsar supports message ack with subscription groups. The worst case with Pulsar is you're storing the entire retention period. Let's say you have a 4 day retention window, to cover an outage happening on Friday and not having to deal with it until Monday. This is pretty typical with what I see in the Kafka world for small-mid size companies who don't want to pay the 1.5x OT on call. So, with Pulsar you're at worst storing the 4 days of data but at best you're only storing the messages within the lag period of all consumer groups acknowledging the message. Now, without getting too deep into Pulsar's feature set even that is a lie because Pulsar has tiered storage as a first class citizen. The messages after the four days could be ship off to S3 if we wanted or even within 1 day depending on our use case and this is all built into Pulsar, no OpEx tooling required. Even access the messages from S3 through Pulsar is abstracted, there's no tooling required to pull them back in if you wanted. Now with Kakfa our worst case is simply 4 days of retention data. This can get very expensive as compute & storage are tied together, it means scaling up all the brokers (even though we don't need the throughput) for the storage increase. Now, yes MSK basically abstracts all this from you but you're paying for it. 6. AWS Managed Service are not equal citizens to EC2 standalone Managed services right now don't fall under the new Saving Plan: https://aws.amazon.com/blogs/aws/new-savings-plans-for-aws-c... This will cost you 30-60% discount on your entire Kafka bill. 7. Excel Life If I look at the numbers for what I'm doing it would have costed ~$4M for Kafka vs ~$1M for Pulsar. |