[imiller]

I wonder how changing the rating system in cases like these from a five star scale to a thumbs up/down system would affect user behavior and system usefulness. It seems like ebay, uber, airbnb, etc all suffer from the same "anything less than 5 stars is effectively negative" issue.

[moojd]

Once you have a large enough userbase the thumbs up / thumbs down method might be the best option. My favorite example of numbered rating systems failing is metacritic where almost every user score is either a 10 or a 0. They have a weird dynamic where users will give a 10 to counteract a low score or a 0 to counteract an inflated score. Giving a 0 or a 10 makes your vote more impactful so why would you score any different?

At the end of the day most people just want to know if people think a service, movie, driver, seller, etc. is worth it or not * . If you need more info you are going to read the reviews anyway.

* An exception might be niche user review communities like myanimelist or rateyourmusic

[KozmoNau7]

Grading scales can work, for instance if people are rating and reviewing products in online shops.

You're looking at a jacket and you see that while some people rate it 5/5, most of the ratings are 4/5 and 3/5, and 85% of the reviewers say they would recommend the jacket to a friend.

So you read the reviews, and gather that most of the medium scores are because the arms are a little shorter than expected and the cut across the back is rather wide. You also notice that everyone is praising the quality of the fabric and worksmanship. Good information, since if you have short arms and the back muscles of a powerlifter, that jacket would probably be a good purchase.

That is good use of a scoring system, but it requires people to care about it and about writing reviews. It works for niche sites, but completely breaks down somewhere like Amazon, or on Uber/Airbnb where you get punished for anything less than a perfect score.

[colinmhayes]

Who looks at the user reviews on metacritic?

[freddie_mercury]

Lots of people?

The difference between the critic rating and the user rating is useful information to have when making a viewing decision. If they are both low, that tells you something. If one is low and one is high, that tells you something.

[ben509]

Everyone who has ratings does it. I used to rate 3 = professional and fully acceptable until I got a call from Dell asking what they had done wrong, and then from Honda. I realized ratings are all inflated and staff are effectively penalized for less than perfect, and since I'm not going to screw someone because I disagree with their rating scheme, I just set 5 to acceptable and go off that.

The sane way to do it would be to ask yes no questions, but that would require thinking.

[buckminster]

So why leave ratings?

Nearly every time I have a bad experience with a company the problem is systemic, but a bad rating will be associated with the random person who served me. So I don't participate. If the problem is particularly bad I find another supplier.

[ben509]

If they did their job, I'll give them the "you did your job" 5 stars.

Even if the company has systemic problems, they still need to pay bills as long as they still have the job.

Finding another company is a smart option. I'd also write to the management to explain why that problem is blocking you from doing business with them, without mentioning who you spoke with.

[snazz]

Ebay has a positive/neutral/negative review system for sellers, right? Does that have any interesting side effects?

[blululu]

This reminds me of the idea of net promoter score in customer satisfaction surveys. It's a variation on the idea that it's better to have imprecise knowledge that is relevant than precise knowledge that is irrelevant. Net Promoter Score replaces a litany of questions about satisfaction with a single question ('would you recommend this product to a friend' (yes/neutral/no -> 1/0/-1)? You take the average of this score ((yes - no) / total) and it corresponds roughly to the health of a product. The basic idea is that asking a bunch of questions doesn't really add much knowledge, and giving more choices and asking more questions can actually lead you down a false path.