[jimbokun]

These guys appear to be having a hell of a lot of fun. Their technique of wrapping a stack from Amazon EC2 through Hadoop all the way up into Clojure, was the kind of thing I wondered about being possible, so it's pretty awesome to hear it is being done and done well by someone. The idea of iterating with Clojure in a REPL on a small dataset to develop or refine an algorithm, then pressing a button and see how it does running on some large dataset on EC2, sounds sublime.

Even if they never release any of the glue code that makes all this happen, just knowing it is possible is very encouraging.

[pskomoroch]

The real world Lisp/Clojure + Hadoop approach is definitely fun to hear about. Also interesting to see a YCombinator team with 8+ people including a domain expert. Mashing up 4+ messy data sources is tough to do. Very unconventional on many fronts.

[jaf12duke]

It's totally fun--we love it. Everyone has a role that they own and we trust each other to execute. Stay tuned--this is just a small slice of what's coming...

Jason (@FlightCaster)

[technomancy]

> The idea of iterating with Clojure in a REPL on a small dataset to develop or refine an algorithm, then pressing a button and see how it does running on some large dataset on EC2, sounds sublime.

Unfortunately it usually works out to spending a few hours iterating on the functions in the REPL till it works great and then spending a week battling with Hadoop to make it actually run the way you intended it to. (no exaggeration here.)

Though we've been working with raw Hadoop; it sounds like Cascading makes it much less painful. We'll see how that goes.

[bradfordcross]

True dat. Spoken like a true clown zen master. It wouldn't be clowncomputing if we didn't have red rubber noses and funny rainbow hair.