[mack73]

Sure, but what I was saying was that what good is a new ranking method, when you only have at your disposal the same set of metrics as the method you are trying to replace? A new ranking would quite often mean adding new metrics. For example, when Lucene when from tf-idf to bm25 they added lots of new metrics to be able to cater for the new algorithm.

[ganeshkrishnan]

did lucene go from tf-idf to okapi bm25? Surprising. Need to research it up.

We use tf-idf too but augment with page rank and clustering. gets more relevant docs