[hkmurakami]

Seeing some of my favorite hole in the wall restaurants get bad reviews and seeing some bland and uninteresting (but good) places get great reviews on Yelp has made me remember Malcolm Gladwell's commentary on people's preferences for three distinct flavors of spaghetti sauce.

"If you sifted carefully through the data, though, you could find patterns, and Moskowitz learned that most people's preferences fell into one of three broad groups: plain, spicy, and extra-chunky"[1]

Which tells me that since Yelp tries to aggregate everyone's taste into one unified rating metric, the tastes from which Yelp ratings are derived are quite likely to be different from those of my own.

Thus I really can't trust Yelp ratings, even if there were no 'gaming' or 'extortion' going on in their reviews.

[1] http://www.gladwell.com/2004/2004_09_06_a_ketchup.html

[grey-area]

This is a really interesting point, unfortunately the counterpoint to this is that a complicated rating metric is not successful either. People don't want to spend ages filling in some huge questionnaire and if you start trying to introduce complicated systems of combination star ratings like the zagat ones for example, it just bamboozles both reviewers and readers, and still has the same issues as a simple 5 star rating does - it's too rough a tool and tries to decide in advance on a procrustean taxonomy into which all decisions are forced.

I think the most useful reviews are textual ones which set out the interests and prejudices of the writer (sometimes inadvertently), and elaborate on why they like or don't like this particular place/thing, which can give you clues about their lifestyle, tastes, and even whether they were hired by the company to write this review in the first place! I'm always suspicious of reviews which say something is great and then list several selling points as if they were read from a brochure - not many people write like that without prompting and it's a good way to find fakes.

Unfortunately another problem as outlined in that article is that even if you managed to completely represent the views of others, you'd find that many of them were simply uninteresting to you, as they are based on false premises or different tastes. That's where social rating systems become more important; where you can decide whose ratings you trust (not necessarily friends), and use those to make decisions - I still think you need the complexity of written reviews to really flesh out people's opinions though. We have a few ideas on this over at coolplaces, but have not yet had time to take them anywhere. Trust does require that you identify with a person's tastes, and also that you know they are real, so using a network of like-minded people built up over time can help mollify the concerns you have about yelp reviews.

Thanks for the great article linked, I've not had time to finish it but it really does cover this subject in depth from a different perspective.

[hkmurakami]

Thanks for the thoughtful reply. I myself rely on RL friends, which is quite similar to what your proposed solution is actually doing :).

(My friend used to work in the food/restaurant business and had hundreds of restaurants in the area as customers, plus I've known him for 15+ years now so he is an ideal guy to ask for recommendations for me)

I'll also point you to Gladwell's TED talk on the same subject. He's a wonderful storyteller with a very distinct style: http://www.ted.com/talks/malcolm_gladwell_on_spaghetti_sauce...

[joshhart]

Yelp has looked at using collaborative filtering to improve recommendation accuracy (this technology could handle the spaghetti case you mentioned), and it looks like someone (probably an intern) published a paper on it. http://cs229.stanford.edu/proj2009/Fennell.pdf

Looks like they didn't have sufficient data 3 years ago, but I imagine the prospects might be a little better now.

[fsckin]

What a great quote.

I have a similar pet peeve with brownies. There are three major types: Cake, fudge or chewy. I do not prefer any type of cake brownie.