[gruseom]

How do you tell what kind of job you have without running it a hundred times?

Or do you mean run small portions of a large job before deciding where to commit the rest?

[pjscott]

I meant each type of job; sorry for the ambiguity. So, for example, if you have a job type that is "look over a chunk of text and return term frequencies", you could test this on a few hundred chunks of text, and then use this profiling information to guide the scheduling any other time you run a job of this type.

[cma]

Google has this covered in spades: http://research.google.com/pubs/pub36575.html