Hacker News new | ask | show | jobs
by pps43 3338 days ago
But that's not how you should measure it. You goal is not to minimize MSE. Your goal is to rank movies in a way that users like.

So the test would be to randomly split users into test and control, show ranking based on Bayesian averaging to control, show ranking based on James-Stein or some other method to test, measure some metric of user happiness (a different hard problem, click rate on top titles?), then do the comparison.