[usgroup]

Lol was hoping it was a combination of awk and paste :)

That always makes me chuckle.

Honestly though ... Jenkins + bash + cloud storage and you'll be surprised at how many big data problems you can solve with a fraction of the complexity.

[pmlnr]

https://aadrake.com/command-line-tools-can-be-235x-faster-th...

Discussion is here: https://news.ycombinator.com/item?id=8908462

[makapuf]

Pardon my ignorance but what would you use Jenkins for ? Scheduling ?

[tetha]

Jenkins in such a setting gives you two good things: i) scheduling. and ii) access control. The ability to give random dude X the ability to trigger computation Y, Z, and A, without the ability to change said computations.

[usgroup]

Triggering jobs more generally (schedule or push notification) and splitting things into jobs and/or pipelines.