Hacker News new | ask | show | jobs
by jamii 4665 days ago
This idea - having locally unrestricted components connected by a restricted, declarative system - seems to keep popping up and never quite taking off. Hopefully sooner or later someone will spin it just right.

Some related projects:

Opis - http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.167.... - pure functions connected by dataflow combinators (with time travel debugging and model checking at the dataflow level)

Bloom - http://www.bloom-lang.net/ - temporal logic for coordination, message passing for communication with stateful endpoints

Flask - http://dash.harvard.edu/handle/1/2797447 - a c-like language for local sensor computations with a haskell dsl for specifying communication patterns across the sensor network

Then of course there is all the FRP work (eg http://www.haskell.org/haskellwiki/Functional_Reactive_Progr... http://docs.racket-lang.org/frtime/)

Despite all the interest, overlog (the predecessor to bloom) is the only language I've seen so far that has any convincing demos (hdfs and hadoop in 8kloc). I'd love to know if anyone is actually using this sort of stuff in production or, if not, why not? What goes wrong when you try to scale this up?

EDIT: I forgot to mention Mozart/Oz, one of the most interesting dataflow languages and one that explicitly talks about layering progressively less and less deterministic systems of concurrency - http://www.mozart-oz.org/features.html

8 comments

There's also Orc: http://orc.csres.utexas.edu/

Regarding "not taking off": this model is actually very common in the vfx industry. Most high-end content creation tools (Nuke, Maya, Houdini, etc) have a node graph which you assemble in the tool, where the nodes define processing operations & are usually plugins written in c++ or something like it. It's not a language per-se and the semantics are tightly bound to the individual tools rather than being independently specified, but the underlying idea is the same.

There has been some interesting work from Johnathon Edwards @ http://www.subtext-lang.org/
I've had his blog bookmarked for a while but never got around to actually reading it. I'll have to take a look, thanks.
Weird that nobody mentioned spreadsheets until now... spreadsheets are one of the possible data flow implementations.

Probably few people know but Microsoft Excel has some optimizations like "multithreaded recalculation" http://msdn.microsoft.com/en-us/library/office/bb687899.aspx and "high-performance Excel based applications in financial services" http://msdn.microsoft.com/en-us/library/bb887539.aspx

I included extra resources at the end of my articles on the subject: http://blog.databigbang.com/category/egont/

Also http://www.kamaelia.org/Home.html developed by BBC R&D using Python. It seems to have been quiet for a while now though.
From ancient history (although explicitly based on Hoare's CSP) there was Occam for the Transputer.
Found another one.

http://meemoo.org

Quartz Composer?
BizTalk Orchestrations would probably fit. I also remember an old robot programming game.
Thanks for the list of related projects!