[pps43]

The choice of m and C need not be exact. It is enough to choose them so that

1. If there are no ratings, Bayesian average is close to overall mean, and

2. If there are many ratings (how many depends on how big the site is), C and m do not affect the result much.

You probably can do a little better if you have a lot of data and ability to run A/B tests, but for vast majority of cases pseudocounts work just fine.

[csaid81]

Got it. Thanks for the clarification. In that case I would think that James-Stein / Buhlmann / BLUP is a better approach, since it is just as easy to implement and the amount of shrinkage is optimally chosen based on the data, rather than on guesswork. In fact it may be more easy because no guesswork is required.

It would be interesting though to have people try to guess suitable values of m and C and then see how close their MSEs get to the James-Stein MSE. I suspect that some people's guesses would be meaningfully off target.

[pps43]

But that's not how you should measure it. You goal is not to minimize MSE. Your goal is to rank movies in a way that users like.

So the test would be to randomly split users into test and control, show ranking based on Bayesian averaging to control, show ranking based on James-Stein or some other method to test, measure some metric of user happiness (a different hard problem, click rate on top titles?), then do the comparison.