[apurva]

One question that definitely has plagued me for a while is that of evaluation schemes of recommender systems... On multiple occasions I have designed recommender systems that give a fairly good user experience, but I just have not been able to quantify these results... any pointers on that will be really appreciated.

[ogrisel]

xlvector has a blog on recsys that talks about a good but overlooked feature of those systems from a user point of view: the ability to have the user make fortunate discoveries while looking for something seemingly unrelated:

+ http://xlvector.net/blog/?tag=serendipity

I have no easy answer on how to quantify that myself but I guess the papers quoted by xlvector must be a good place to start.