[deniscoady]

Disclaimer: I work for Redpanda and formerly Cloudera.

I've worked with Apache Kafka at massive (50+ Gbps) scales. It's a proper nightmare. When it breaks – it breaks fast and violently.

But the problem is that Apache Kafka (and more modern Kafka-compatible alternatives like Redpanda < obligatory mention) solve a need for a durable streaming log that other systems cannot offer. The access patterns, requirements, use cases, ecosystem, etc, are different from those of traditional databases and require a proper streaming solution.

Streaming from a traditional database is kinda a solved problem. Why not just use a managed Kafka provider with a change data capture (CDC) capability if you don't want to deal with Kafka yourself? At least then you get to use all of the tools in the vibrant Kafka ecosystem.

[galeaspablo]

Hey Denis, I haven’t run into you before. But hi, this is Luis, Ambar’s founder. Nice to meet a fellow data streamer.

When I started writing Ambar I thought streaming from a database was a solved problem. But in operational use cases where ordering and delivery guarantees are assumptions developers need, it isn’t a solved problem. The first version of Ambar was just Debezium under the hood, but guess what, it failed and failed hard. Like you described Kafka. Hence we built Ambar :)

FYI we’ve considered using Redpanda under the hood instead of Kafka, but didn’t dare make the jump yet.

[deniscoady]

Ah okay, so is Ambar more of a way to finally replace Debezium then?

[galeaspablo]

Yes, for operational use cases. Eg event driven microservices communication. Keeping in mind we replace the sink as well, which allows us to do cool things such as https://ambar.cloud/blog/optimal-consumption-with-adaptive-l...

For analytics (eg copy your PG database to Snowflake), Debezium is still relevant.