[jpgvm]

i.e this is a crappy tool but it's best tool we have come up with for this problem thus far.

There are lots of such tools which are used begrudgingly by people that have an intuition for the fact it can be done better but not the concrete idea and/or time to implement it.

[dekhn]

it wasn't a crappy tool (mapreduce was amazing) but it definitely was an impedence mismatch for this particular job. Later, we tried to get SIbyl to move to the underlying compute engine that Flume was built on top of it but it turned out to be more profitable to just let it die slowly.