Seeing some of my favorite hole in the wall restaurants get bad reviews and seeing some bland and uninteresting (but good) places get great reviews on Yelp has made me remember Malcolm Gladwell's commentary on people's preferences for three distinct flavors of spaghetti sauce.
"If you sifted carefully through the data, though, you could find patterns, and Moskowitz learned that most people's preferences fell into one of three broad groups: plain, spicy, and extra-chunky"[1]
Which tells me that since Yelp tries to aggregate everyone's taste into one unified rating metric, the tastes from which Yelp ratings are derived are quite likely to be different from those of my own.
Thus I really can't trust Yelp ratings, even if there were no 'gaming' or 'extortion' going on in their reviews.
This is a really interesting point, unfortunately the counterpoint to this is that a complicated rating metric is not successful either. People don't want to spend ages filling in some huge questionnaire and if you start trying to introduce complicated systems of combination star ratings like the zagat ones for example, it just bamboozles both reviewers and readers, and still has the same issues as a simple 5 star rating does - it's too rough a tool and tries to decide in advance on a procrustean taxonomy into which all decisions are forced.
I think the most useful reviews are textual ones which set out the interests and prejudices of the writer (sometimes inadvertently), and elaborate on why they like or don't like this particular place/thing, which can give you clues about their lifestyle, tastes, and even whether they were hired by the company to write this review in the first place! I'm always suspicious of reviews which say something is great and then list several selling points as if they were read from a brochure - not many people write like that without prompting and it's a good way to find fakes.
Unfortunately another problem as outlined in that article is that even if you managed to completely represent the views of others, you'd find that many of them were simply uninteresting to you, as they are based on false premises or different tastes. That's where social rating systems become more important; where you can decide whose ratings you trust (not necessarily friends), and use those to make decisions - I still think you need the complexity of written reviews to really flesh out people's opinions though. We have a few ideas on this over at coolplaces, but have not yet had time to take them anywhere. Trust does require that you identify with a person's tastes, and also that you know they are real, so using a network of like-minded people built up over time can help mollify the concerns you have about yelp reviews.
Thanks for the great article linked, I've not had time to finish it but it really does cover this subject in depth from a different perspective.
Thanks for the thoughtful reply. I myself rely on RL friends, which is quite similar to what your proposed solution is actually doing :).
(My friend used to work in the food/restaurant business and had hundreds of restaurants in the area as customers, plus I've known him for 15+ years now so he is an ideal guy to ask for recommendations for me)
Yelp has looked at using collaborative filtering to improve recommendation accuracy (this technology could handle the spaghetti case you mentioned), and it looks like someone (probably an intern) published a paper on it. http://cs229.stanford.edu/proj2009/Fennell.pdf
Looks like they didn't have sufficient data 3 years ago, but I imagine the prospects might be a little better now.
A company that is constantly being sued for extorting businesses into subscribing so that those businesses can suppress bad reviews is now calling out businesses who purchase good reviews.
Could you substantiate that claim? I did a few Google searches, and I find your remarks to be, well... just wrong.
As far as I can tell, there was a class action lawsuit against Yelp, which was dismissed, and a couple of other lawsuits from single business owners. I would imagine that any company the size of Yelp is going to get sued a few times, and I imagine a lot, if not all of those, are just bad small businesses.
Are you just scurrilously slinging mud, based on a single headline you read sometime in the past?
I find Yelp to be quite useful, and the people who I have met that worked there were good, hard-working souls.
So let me understand this. You have a great small business where your customers are really happy. They decide to give you a positive review on Yelp. Boom, you get calls from Yelp to pay for advertising. You refuse to do so and boom, all your positive reviews are removed. And they add/fake negative reviews OR leave the negative reviews on from customers ?
I imagine you don't trust any major review company then. The BBB must be a bunch of scumbags too, if your metric is what a google search looks like when attaching the word extortion to their name: https://www.google.com/search?q=bbb+extortion
Yelp needs negative reviews. This is not a site comprised of people with impeccable taste that can rival any restaurant critic or ones's own sense for restaurant choices, based on experience, or word-of-mouth (spoken word). This is yet another way to use the negativity of the web (XYZcompanysucks.com) as a form of blackmail. What a joke. The company that needs to be "outed" is Yelp. If all the accounts are true, these guys are crooks.
Anyone who bases all their dining decisions on what people post to sites like Yelp is not going to have as rich an experience as people who mastered the "old ways" (i.e. personal knowledge not acquired online). Why? Because Yelp is _primarily_ for people who want to complain. And _primarily_ for people who have no taste or worldly experience but who want to post things to the web (tech savvy). There are exceptions, but c'mon, let's be real. The web is the ultimate gripe channel.
Don't believe me? Try basing all your decisions on Yelp. You will not eat as well the the guy who has a sincere interest in restaurants and understands how to use the "old ways". There's certainly potential for such knowledgeable people to come together at a site and produce something amazing: balanced, informative, _informed_ reviews. The web is an amazing instrument for information excehange. But Yelp is not such a site. Their "business" relies lots of complainers who usually (have tech savvy but) have no taste, and on the restaurateur's fear of a fool with a megaphone.
Yelp's strategy could backfire. Now all a competitor has to do is put ads on Craigslist or Mturk begging for reviews for all of its competitors, knowing Yelp will see them and penalize them.
Yelp is going to find themselves boxed in on this. Google has already been down this road with links, and they've taken the stand that "no link can hurt you" because it's too easy for competitors to manipulate that.
"If you sifted carefully through the data, though, you could find patterns, and Moskowitz learned that most people's preferences fell into one of three broad groups: plain, spicy, and extra-chunky"[1]
Which tells me that since Yelp tries to aggregate everyone's taste into one unified rating metric, the tastes from which Yelp ratings are derived are quite likely to be different from those of my own.
Thus I really can't trust Yelp ratings, even if there were no 'gaming' or 'extortion' going on in their reviews.
[1] http://www.gladwell.com/2004/2004_09_06_a_ketchup.html