Hacker News new | ask | show | jobs
by moojd 2300 days ago
Once you have a large enough userbase the thumbs up / thumbs down method might be the best option. My favorite example of numbered rating systems failing is metacritic where almost every user score is either a 10 or a 0. They have a weird dynamic where users will give a 10 to counteract a low score or a 0 to counteract an inflated score. Giving a 0 or a 10 makes your vote more impactful so why would you score any different?

At the end of the day most people just want to know if people think a service, movie, driver, seller, etc. is worth it or not * . If you need more info you are going to read the reviews anyway.

* An exception might be niche user review communities like myanimelist or rateyourmusic

2 comments

Grading scales can work, for instance if people are rating and reviewing products in online shops.

You're looking at a jacket and you see that while some people rate it 5/5, most of the ratings are 4/5 and 3/5, and 85% of the reviewers say they would recommend the jacket to a friend.

So you read the reviews, and gather that most of the medium scores are because the arms are a little shorter than expected and the cut across the back is rather wide. You also notice that everyone is praising the quality of the fabric and worksmanship. Good information, since if you have short arms and the back muscles of a powerlifter, that jacket would probably be a good purchase.

That is good use of a scoring system, but it requires people to care about it and about writing reviews. It works for niche sites, but completely breaks down somewhere like Amazon, or on Uber/Airbnb where you get punished for anything less than a perfect score.

Who looks at the user reviews on metacritic?
Lots of people?

The difference between the critic rating and the user rating is useful information to have when making a viewing decision. If they are both low, that tells you something. If one is low and one is high, that tells you something.