| I'll meet you halfway. It's a terrible comment about a terrible article. I agree that it's laudable to try to invent an improved version of online ratings, but this article wasn't effective in convicing me that they've succeeded. Their argument that historgrams are just.awful. (I didn't care for the extra periods) seems to have two components:
1. asserting that historgrams are bad
2. showing us 3 histograms and saying they tell you the same thing about all 3 movies, when in fact there is a very clear and important difference between the histograms. Its completely obvious from inspection ("people are really good at seeing patterns") that Starship Troopers has a much lower percentage of 5 star ratings than the other two, and a much higher fraction of 0 star ratings. It also appears to me that the Fifth Element has a higher fraction of 4 or 5 star ratings, and is probably the most apprciated of the 3 films, although Blade Runner is fairly close. If you are going to cherry a set of 3 specific films to make your point, you should be sure to at least pick 3 films that support your point instead of refuting it. We then learn about their hypothesis that while 5-star rating system sucks, a system that relies on two correlated 5-star ratings is great. They demonstrate this by using the two question system to draw the exact same conclusion I drew from the histograms of the 1 question ratings. I would've liked some sort of objective attempt to compare the two rating systems. Perhaps it would be possible to measure how frequently the two question system leads people to make a better choice than the one question system, or at least some sort of statistical wonkery that would purport to show me that the two question system in practice draws more distinctions than the one question system. Unfortunatley we only get this one rather uninspired example ("watch this if you’re in the mood for something really good"). They also didn't address why "would you re-watch this film" is a better choice than any other second question. There are attempts to justify it being a good question, but no real evidence that other questions were tried and didn't perform. Finally, the thing that really irked me was that this proposed system doesn't seem to do anything to address most of the actual problems with the regular 5 star system, namely that people who feel really strongly about something are more likely to rate so that most ratings tend towards the extremes, and that without context we have no idea why someone rated something a 5 instead of a 1. Those problems now exist along two dimensions instead of one. I see this less as an article and more as an advertisement hitching a ride on an xkcd comic. |