|
I have many problems with this analysis, perhaps enough to write my own blog post, but I am lazy so I will just outline my disagreements here (at least for now). First: I claim that what we seek in a movie rating is information about whether we will like the movie, and that this can be formalized as the expected KL-divergence (information gain) between the Bayesian posterior distribution (probability of enjoying the movie conditional on its rating) and the prior distribution (probability you would enjoy a randomly selected movie). Of course, this will depend on your taste in movies, especially how much it correlates with others. But, we can _bound_ it by taking the Shannon entropy of the rating distribution: there is no way we can get more information from a rating than this! It is this bound that allows us to penalize the distributions that are heavily biased towards one side of a discrete scale, like Fandango. However, the "ideal" shape in this context is far from a Gaussian - it is uniform! The uniform distribution can also be justified as being calibrated such that the quantile function is linear - a score of 90/100 from a uniform distribution means a 90th-%ile movie. Determining a quantile is often a transform we try to perform intuitively on ratings so such a transform being trivial seems useful. Second: The Gaussian distribution does not have bounded support! That is, a rating scheme with what you claim as the "ideal" distribution would have _some_ ratings with values that are negative or otherwise "off the scale". Not so ideal! If you wanted to model movie-goodness on an unbounded scale such that a Gaussian would have sense, then you should transform that scale into a bounded scale, eg with a logistic function, yielding an "ideal" shape of a logitnormal distribution, which incidentally can fit the strange bimodal Tomatometer distribution quite well. Even if you specifically wanted a unimodal, bell-shaped distribution, at least pick a bounded one like the beta distribution. Third: setting aside which distribution you want to penalize distance from or why, dividing the space into three arbitrary intervals to facilitate the comparison seems ridiculous. There is already a perfectly good metric on probability distributions, the mutual information. |
http://blog.moertel.com/posts/2006-01-17-mining-gold-from-th...
This was about a decade ago, so I'd expect the resulting decoder ring to be somewhat miscalibrated for today's movie ratings. But the same process would be straightforward to apply to a more up-to-date data set of ratings.