Hacker News new | ask | show | jobs
by jacques_chester 2309 days ago
I think the main step forward is that it has an efficient means for calculating what views change when a new datum arrives. Which means that it could, in theory, hold a very large amount of views for relatively low overhead.

It's a surprisingly difficult problem. I wouldn't roll my own.
1 comments
benesch 2309 days ago
Yes, absolutely. Entire dissertations have been written on the topic: https://dash.harvard.edu/bitstream/handle/1/14226097/KATE-DI...

Efficiently maintaining views over arbitrarily complex computations is one of the two hard problems in computer science for a reason [0].

[0]: https://martinfowler.com/bliki/TwoHardThings.html

link
jacques_chester 2309 days ago
Never let HN haters (among whom I am frequently numbered, it ought to be noted) get you down.

After skimming a bit of the differential dataflow writing I am really impressed. This is deep computer science doing what it does best, which is to do much more with much less.

link
benesch 2309 days ago
I’m glad to hear it! One of my favorite ways to view Materialize is as bringing differential dataflow to the masses. Differential dataflow is deep, elegant stuff, but it requires some serious CS chops to grok.

SQL is lacking in elegance but abundant in popularity. Building the SQL translation layer has been a fun exercise in bridging the two worlds.

link