[talktime]

Reputational Review Network - A review network where users can rate local restaurants, services, goods and other reviewers' credibility. Ratings are personalized based on your own reviews and who you find credible - this gives a user an incentive to keep their reviews honest.

[bostik]

I'd like a rating system that intelligently weighs the reviewers and provides the overall grade after adjusting for individual review points.

Examples:

- if a person only posts 5-star or 1-star reviews, they are polarised and/or punitive. Any and all reviews from such users should be weighted accordingly - 5 stars clearly means "thumbs up, for whatever reason" and accordingly 1 star means "thumbs down, for whatever reason".

- if a person posts a fair selection of review scores, they are likely more thoughtful. Individual review scores from these users are more accurate.

- if a person posts mostly 2/3/4-star reviews, then any 1/5 star review is likely to carry a lot more weight.

To make score manipulation slightly harder, some kind of meta moderation would need to be in place too. Preferably on a per-category basis, so your reviews might have "good karma" on some topics and "bad karma" on others.

[jon_richards]

One complaint I've heard is people looking for restaurants and finding 4-5 star rated McDonalds. Seems ripe for PCA or something. There are a ton of examples of this sort of analysis too, especially since Netflix made their datasets available. Here's the first paper I found: http://www.lkozma.net/mlsp09binary.pdf

[chrisfosterelli]

I think absolute ratings for restaurants in general tend to be garbage, since a rating is skewed to the average expectations of the guests.

The 5-star McDonalds may be a really good McDonalds that's very clean and consistent, but the $50/meal ocean-side bar only has 3-stars because the steak is sometimes rare instead of medium-rare. People are rating relative to their expectations of the restaurant, so someone who doesn't share those same expectations will be confused that a dollar cheese burger is 5 stars but a prime rib steak is 3 stars.

[jon_richards]

Yeah, that's why I recommend PCA. It works by finding groups of agreement. You can then classify people by how much they resemble each group and predict how they will react to a new thing by how their groups have reacted to it. One criticism is that using it can easily lead to constructing echo chambers, but that's exactly what you want for reviews. If you care about how a steak is cooked, here is how other people who care about how a steak is cooked review this restaurant.