Hacker News new | ask | show | jobs
by vosper 4124 days ago
Hi Michael, thanks for your work creating Onyx - it looks really cool.

I can infer two of your frustrations with Storm from the above post: that Storm was too closed, and it's information model didn't span across languages very well. If you have the time, could you elaborate on these pain points, and any others that you found?
1 comments
XPherior 4124 days ago
I'll paraphrase a few snippets from my own documentation to answer these questions. Happy to comment more if needed.

Information models are often superior to APIs, and almost always better than DSLs. The hyper-flexibility of a data structure literal allows Onyx workflows and catalogs to be constructed at a distance, meaning on another machine, in a different language, by another program, etc. Contrast this to Storm. Topologies are written with functions, macros, and objects. These things are specific to a programming language, and make it hard to work at a distance - specifically in the browser. JavaScript is the ultimate place to be when creating specifications.

Further, the information model for an Onyx workflow has the distinct advantage that it's possible to compile other workflows (perhaps a datalog) into the workflow that Onyx understands.

See https://github.com/MichaelDrogalis/onyx/blob/0.5.x/doc/user-... for a continued explanation of why Onyx is more of an "open" concept.

link
pixelmonkey 4124 days ago
Michael, can you explain this more? "[Storm] Topologies are written with functions, macros, and objects. These things are specific to a programming language, and make it hard to work at a distance -- specifically in the browser. JavaScript is the ultimate place to be when creating specifications."

I don't really get it. Storm Topologies are built in Java or Clojure using a builder interface, but the data structures for topologies themselves are actually DAGs that serialize using Thrift. It's true that this is a bit heavy-weight compared to something like JSON or EDN, but offering an alternative is a discussion in the community right now. What would your ideal representation of topologies be, actually?

link
XPherior 4124 days ago
I wasn't aware that they're Thrift serializable - that's cool, and offers roughly what Onyx does in terms of its workflow representation.

Onyx goes a little further though in terms of its catalog. I wanted more of the computation to be pulled out into a data structure. That includes runtime parameters, flow, performance tuning knobs, and grouping functions. All of these things are represented as data in Onyx. It's a little harder, at least in my experience, to do these things in Storm.

link