Hacker News new | ask | show | jobs
by hamilyon2 3710 days ago
Luigi does decent job. It is relatively easy to start with and powerful enough to do almost anything
1 comments

I've been using Luigi for a few months, with no complaints. It supports running Python jobs on Hadoop and Spark, but it's not really a MapReduce framework unto itself.

However http://discoproject.org/ might be worth a look as a standalone alternative.

I have used Disco extensively in the past, nothing but good things to say about it. Fast job launch, easy to write, the DFS has been stellar. This was only using Python for job code.