[Kapow]

Would this be implemented as only showing pairs within a certain threshold of difference, or would it choose an application purely at random and then the application with the closest score (or even 50/50 the next highest or next lowest)? The former seems like it could lead to cases where applications end up in a void between clusters and stop appearing in votes, preventing late voters from giving an opinion on them.

Is there a name for this kind of voting process?

[zaguios]

To ensure each application gets the same amount of views (with the exception of the top ranked and bottom ranked applications) you can pick the next highest/lowest and it will work out fine. My concern with this implementation was that it might be possible for the same applications to be compared multiple times at the high and low ends of the bell curve where elo becomes more spaced out. A better solution might be internally listing all the applications by their elo, selecting an application and random, and then selecting from an application within 5 ranks of itself. I.E. App #342 is randomly chosen and then an application between #337 and #347 will be randomly chosen to compare it with. I am not aware of any name for this voting process.

Each potential implementation has slightly better/worse corner cases, but the general quality of the ranking should still be extremely accurate regardless of the particular implementation. It takes surprisingly few comparisons for items to find their appropriate ranks.