Hacker News new | ask | show | jobs
by SamBam 3222 days ago
That's an interesting hypothesis, but I'd want to see more evidence that "a reviewer who believes its true rating should be 3 stars is motivated to give it 1 star."

Surely you can't prove this simply by noticing that there are many 1- and 5-star reviews, as there could be many other reasons for that. One obvious one: people who strongly like or strongly dislike a product are more likely to take the time to review. I personally have never felt the need to review something if I felt "meh" about it.

One sample study might be to see how people's ratings change if they have a chance to see the average rating first or not, but that would be a tricky study as you'd need to get people to buy something without seeing the ratings.

4 comments

That's a good alternative hypothesis. It could also be that people's experience with a product really is bimodal: if I order an alarm clock that works as advertised, it is easy to get 5 stars, if it doesn't work at all, it's 1 star. Your explanation works better for why the distribution persists in books and movies though.

In any case, I find the mechanism design angle interesting regardless of the behavioral angle :)

I've often hypothesized that most people are more likely to leave a negative review when they are upset, than a positive review when they like something. It certainly holds in my case: I nearly never review things, because it's a giant hassle, so I have to feel really strongly to be willing to spend the time on a review.
You're right. It's been proven a bunch of times that people are more likely to leave negative reviews than positive ones, or to remember bad experiences more overall.

Zendesk actually did a survey on this back in 2013:

http://cdn.zendesk.com/resources/whitepapers/Zendesk_WP_Cust...

And American Express found something similar in their Global Customer Service Barometer survey as well:

http://about.americanexpress.com/news/docs/2014x/2014-Global...

It makes sense - a product doing what it says it will doesn't stand out / incentivise a review, whereas a bad product makes you want to caution others / get some sense of justice on the company (not sure of a better way to word that).
> That's an interesting hypothesis, but I'd want to see more evidence that "a reviewer who believes its true rating should be 3 stars is motivated to give it 1 star."

I doubt it's thought about in precisely those terms, but I've certainly thought "that's overrated/underrated" before, and had that affect my rating.

>I personally have never felt the need to review something if I felt "meh" about it.

Or for that matter if I got a simple item and it works as expected. What's an Amazon Basics HDMI cable supposed to do? Does 5 stars mean that it turns HD into 4K through magic? But if it does what an HD cable is supposed to do (and seems well-built) I guess I should give it 5 stars?

There's a fair amount of research on this, although I don't have a link to any public results offhand. But I have sat in on several UX experience sessions with users being interviewed about their motivations for rating, and "fixing" a perceived bad average rating comes up repeatedly.

I'm fairly sure at one point years ago I did see some data showing that rating distribution changes depending on whether the rater sees the average rating first, but it was a long time ago and I don't recall the specific differences now. In practice you're right that this isn't feasible for most use cases.

I do know that the rating distribution is strongly bimodal - on a 5 star scale I think 80+% of ratings will be either 1 or 5 star. Mostly 5 stars - IIRC they were around half of all ratings.