Hacker News new | ask | show | jobs
Differential Dataflow for the Masses (github.com)
42 points by rebanevapustus 592 days ago
4 comments

Also, I never fully grokked Differential Dataflow but I get DBSP so I would love an explanation of how Differential Dataflow differs from DBSP.
https://github.com/brurucy/pydbsp/issues/6 it's been answered! (to the best of my knowledge)
Love this project!

Since "Differential Dataflow" has a specific meaning though and this isn't it, I wonder if a different title wouldn't be better?

"DBSP for the masses" presumes someone already knows what DBSP is so that doesn't work.

What about "Incremental Dataflow for the masses" or "Incremental Dataflow in Python"?

Incremental Dataflow for the Masses sounds much better :)
Electric Clojure v3 also implements differential dataflow. [0]

[0] https://www.youtube.com/watch?v=mP9e_FDjHLU

Electric Clojure v3 cannot be "Differential Dataflow" because it *seems* to be restricted to DAGs only. Differential Dataflow is highly expressive, it supports iteration.
(electric founder here) we are lowercase differential dataflow my friend. Fwiw our model does support cycles and iteration, we simply didn’t bother to formally reify cycles yet, because we have a functional effect system under the hood and using that to implement cycles by userland side effect does the right thing, is perfectly efficient, and has very little downside.
Thanks for clarifying it!

I checked some of your videos, and I couldn't quite grasp how much of (capital) DDflow do you borrow.

What does "our model does support cycles and iteration" mean? Do these also have the incremental semantics that guarantee bounds over the size of updates?

This seems to make the mistake of identifying a vector with a stream.
In this implementation streams are not vectors nor lists. They are ordered hash maps.
Just pointing out what the theoretical documentation says. A stream should have operations for empty, current, and next. You don't want implementation details leaking into the abstract model.