Hacker News new | ask | show | jobs
by Confiks 3205 days ago
Okay, so, what about the New York Times:

https://www.confluent.io/blog/publishing-apache-kafka-new-yo...

1 comments

As the first comment [1] says, it's a silly architecture for them and completely unnecessary.

All modern databases are essentially the same: they receive changes, save them to a write-ahead log or journal, then materialize the data into indexes and tables. This is the event-sourcing + compaction + materialized views model that kafka/confluent keeps pushing but inside a single piece of software that does all the hard things for you with arbitrary queries, reliable backups, etc. There's a reason why this is the common model for decades, there's nothing new here.

Splitting up all that logic into separate event streams, processing pipelines and datastores is certainly valid but only really necessary at the 1% of companies that operate at that scale. For most people, a relational database running on a mid-range server is plenty of power and far better long-term storage.

1. https://news.ycombinator.com/item?id=15184640