[samsquire]

Thanks for this interesting post.

Intuitively, relational algebra compresses enumeration over data in time that a CPU executing billions of cycles a second can feasibly and efficiently traverse and execute against many collections of millions or billions of records in human perceivable time thanks to indexes.

I've been trying to think of systems communicating with eachother as parts of a relational model in the sense we can model system behaviour as a series of events and a join is a communication between components.

I would love to talk about this with people.

[seanhunter]

Some years back I spent a weekend and built a "good enough to prove to myself" version of this in Haskell that implemented the main relational operations of projection, selection and cartesian product.

The basic idea was based on the "stream fusion" papers. So the relation was a stream in the stream fusion sense so it was pretty trivial to implement the normal relational operators in that paradigm. Changing this type of system to work on "events" as input would be pretty trivial.

The one thing I never managed to get to work was the actual "fusion" compiler hint. I kept trying variants of what he did in the paper but ghc just refused to optimise my stream/unsteam unstream/stream pairs away because it had already done some rewriting to them. I couldn't figure out how to apply the optimisation early enough to be effective.

[1] Which are a fantastic read if you're into CS whatever you think of my idea https://www.cs.tufts.edu/~nr/cs257/archive/duncan-coutts/str... and https://www.researchgate.net/publication/220802863_Rewriting...

[samsquire]

Thank you for sharing your thoughts and ideas Sean, appreciated.

I am unfamiliar with the source material and I have recorded the paper you linked to my reading list.

From a description of "stream fusion", it reminds me of Clojure's "transducers".

It also reminds me of Kafka's Table/Stream duality.

Term rewriting is something is really interesting to me.

Communication, protocols and Communicating sequential processes, session types are all ideas I am thinking about and trying to understand.

[ndriscoll]

When working on a system that was overly split into microservices, I once wrote a leftJoin stream operation for Akka, partly to demonstrate how stupid it was that we had split our database apart and now needed to implement a database in our application using http calls, with 1000x worse performance.

[refset]

> a join is a communication between components

Makes me wonder just how far people have pushed Foreign Data Wrappers in practice.