Hacker News new | ask | show | jobs
by anoncept 2098 days ago
Lovely post, and an interesting problem area. Three additional perspectives that have been on my mind in this area:

1. Differential dataflow. Like Adapton and Incremental (which Raph cites), I have found DD to be hugely helpful for understanding which computations can be efficiently incrementalized, and how (including in Rust).

2. Operational transforms. Often times, I am working on UI as part of a larger distributed/networked system. In these cases (especially when I already want the UI to display other people and their actions), the division of responsibility between the UI, the network code, and the various internal models has always felt like a missed opportunity for better and more principled integration / compositionality.

3. Build systems. The problem of figuring out how to bring a UI efficiently up to date in response to some parts of the underlying state changing can also be seen as a build system, or cache coherency problem. From that standpoint, where do the paradigms of the Build Systems a la Carte paper fit into Raph’s pictured theory of reactive UI?

https://github.com/TimelyDataflow/differential-dataflow

https://github.com/mstone/focus/blob/master/docs/ot-theory.a...

https://doi.org/10.1017/S0956796820000088

1 comments

Thank you for your kind words. I agree that all these resources are relevant. I'm familiar with some but will study others more deeply.

Certainly I've spent a lot of time thinking about collaborative and distributed systems, and there is no doubt scope for cross-fertilization of ideas. A huge inspiration (cited in the post) is Martin Kleppmann's automerge, which uses a proxy object to capture an explicit diff from a computation which is expressed using ordinary mutable logic.

Regarding the build system work, I will say this: the build task is inherently graph-like, and a lot of the interesting problems involve choices of sort order to utilize parallelism best; I'm fortunate to have a 16 core machine, and it makes me sad to wait on a single-threaded link step. I think by comparison the reactive UI task is mostly dealing with tree structure, and that's a simplification that could be useful to exploit. It's easier to describe mutation of a tree than general mutation of a graph.

But certainly there are common themes in all these incremental systems, and I continue to be drawn to figuring out the fundamental principles rather than trying to build systems ad-hoc.

Martin’s writing and thinking has been hugely inspirational for me too — thanks for highlighting the automerge work here!

Next, regarding your other thoughts, two notes, one on performance and one on “common themes”.

Re performance: while in my initial comment I linked directly to McSherry & friends’ work on differential dataflow mostly because I value it as a principled and effective framework for incrementalizing many kinds of computation I care about, I would be remiss if I failed to also highlight here their complementary “COST” project of using this framework to demonstrate how carefully engineered (single-threaded!) implementations of graph computations in Rust can often considerably outperform more complex but more wasteful pre-existing approaches (at least in the database world).

(Whether the same flavor of result will also appear over time in the build systems or UI worlds seems like an interesting question for future work!)

Finally, re “common themes in incremental systems”: yes! - this and your point about “fundamental principles” are exactly what I was hoping to suggest by linking to the build systems + OT materials!

Thanks!

http://www.frankmcsherry.org/assets/COST.pdf

https://github.com/frankmcsherry/COST