[netpaladinx]

Ursa published a blog post saying their leaderless, stateless, object storage–based Kafka replacement can reduce costs by up to 95%. Has anyone here tried Ursa in production? How much cost reduction have you actually seen compared to Kafka or MSK in real workloads?

[x0x0]

As near as I can tell, the claims of huge cost savings derive from the difficulty dynamically scaling Kafka and improved multitenancy. So if different pieces of your company each have overprovisioned kafka clusters, they could all move to Ursa and save all the overprovisioning.

I have not tried it, and full disclosure, I really like Kafka: it's one of the pieces of software that has been rock solid for me. I built a project where it quietly ingested low gb/s of data with year-long uptimes.

[davidkj]

The bulk of the cost savings comes from the use of object storage rather than attached disks. This eliminates the inter-AZ networking costs associated with Kafka replication mechanism.

I break all of the costs down in the following e-book. https://streamnative.io/ebooks/reducing-kafka-costs-with-lea...

[x0x0]

So basically Kafka, to provide availability guarantees, requires multi-AZ and the inter-AZ replication gets expensive. And Ursa avoids that by using object storage and probably then just talking inter-AZ?

And while I like Kafka, nobody would claim it likes being scaled up and down dynamically, so probably built-in tolerance for that as well? We ran Kafka on-prem so that wasn't an issue for us, and given the nature of the service, didn't have a lot of usage variance.

This: https://www.youtube.com/watch?v=bb-_4r1N6eg was an interesting watch, btw.

[enether]

Yes, see WarpStream, Confluent Freight and Diskless Kafka for other examples

[sijieg]

We also love Kafka as a protocol. However, the implementation can be evolved to adopt the current cloud infrastructure and and rethought based on the modern lakehouse paradigm. That was one of the reasons we created Ursa.