[andrewprock]

When you say the ranking was quantitative, what sort of quantities did you measure? Did you ever evaluate the quality of your metrics, and how predictive they were for future success?

IIRC, Google did the same thing and concluded their hiring metrics had very little predictive value beyond setting a certain bar. Once you rose to that level, most discriminatory power of the metric evaporated.

If that was the case where you were, then is probably very little predictive value between "top scores" and "median scores".