[thinksketch]

The funny thing about the 5 star rating system is that it's completely the wrong model for many situations in which it's used. It originated from sites like slashdot where, because everyone had to read the same front page of content, it made sense to pick the content democratically using a five star system.

But, systems like netflix that use the five star rating system have it all backwards! Their goal should be to establish the niche genres that attract each user. This goal has nothing to do with brute popularity. Sure you want to assess how much someone likes a movie in order to steer them into the correct genre, but it doesn't make sense to attach an overall popularity rating to the movie.

Given a hugely diverse database such as the netflix movie library, it's ridiculous to assume that individuals will like things in proportion to the average popularity. That's not how taste works. And the weird thing is, there is no reason for these databases to constrain themselves to an averaged popularity index - they're just accustomed to the five star model is all. They should be using a micro-genre mapping scheme that steers you towards clusters of movies that have received attention from users with similar taste.

Meanwhile, by constraining your rating to a discrete number of stars (1,2,3,4, or 5) they are killing the quality of their sample. (See the jellybean guessing experiment in the link below..)

These algorithms should ditch their discrete-value database of how much users say they like something, and instead use some continuous measure of how much attention is spent on each item. Hell, some people love to watch crappy movies and write bad reviews for them. Anyway, for more on this rant, see the link below.. cheers-

http://www.thinksketchdesign.com/2008/05/03/design/algorithm...

[shpxnvz]

Granted, it's been a year or two since I ditched Netflix, but as I recall their recommendation system already ranked the importance of ratings based on how similar the other reviewer's tastes were to yours (I imagine by finding other people who tended to have ranked movies the same as you).

My impression was that they were far more concerned with doing well with recommendations than with the plain rankings.

[thinksketch]

That's true, but their entire model is still built on a database built up of 5-discrete-value ratings. True, it does seem that they place an emphasis on reviewers' similar tastes, but they are still crippled by the poor resolution of their data.

They're spending so much energy tweaking their 5-star algorithm by tiny amounts, but it seems like they'd be much better off investing in a richer database medium - like attention spent browsing various genres on their website while looking for their next movie...

I don't know, it just seems like the five star system is so crude for a company willing to spend millions to improve their recommendation system by even a tiny amount.

"So you like this movie? Like, would you say, "4 stars" like it, or "5 stars" like it?" really? That's what your database is made of? - know what i mean?

[kakooljay]

That is a great point.. imagine a prof issuing grades that way. I'd love to see someone with a 100-point system do some A/B testing to see how much recommendations degrade with a 5-point system.