[elvinyung]

It's less of a message broker and more of a scheduler, although considering that both of them have some kind of queue at its core, the distinction is probably more superficial than it seems (a FIFO job queue is basically a kind of scheduler).

Spark has a scheduler that takes data locality into account (although I don't have a ton of experience with it and have no idea how well it works on a fine-grain scale). That itself is probably inspired from Borg and MapReduce, both of which have some kind of data locality-based job scheduling logic.