[ndm000]

I’ve felt the same way. It’s so inefficient to have two patterns - OLAP and OLTP - both using SQL interfaces but requiring syncing between systems. There are some physical limits at play though. OLAP will always take less processing and disk usage if the data it needs is all right next to each other (columnar storage) where as OLTP’s need for fast writes usually means row based storage is more efficient. I think the solution would be one system that stores data consistently both ways and knows when to use which method for a given query.

[thom]

In a sense, OLAP is just a series of indexing strategies that takes OLTP data and formats it for particular use cases (sometimes with eventual consistency). Some of these indexing strategies in enterprises today involve building out entire bespoke platforms to extract and transform the data. Incremental view maintenance is a step in the right direction - tools like Materialize give you good performance to keep calculated data up to date, and also break out of the streaming world of only paying attention to recent data. But you need to close the loop and also be able to do massive crunchy queries on top of that. I have no doubt we'll get there, really exciting times.

[ndm000]

Completely agree. All of the pieces are there and it's just waiting to be acted upon. I haven't seen any of the major players really doubling down on this, but would be so compelling.