Hacker News new | ask | show | jobs
by mohinish 1436 days ago
Companies have built enormous batch processing systems to transform, format, and store data in systems that are ready to be queried by applications across their organizations. Batch processing is thus a result of the limitations of SQL since the data has to be queried repeatedly to keep it fresh.

What if the same data is processed in real-time as it flows in with applications instantly able to access, minus all the complexity of maintaining a batch processing pipeline? I believe this is going to be the promise of streaming database systems.