[fnordian_slip]

At first your system seemed to me like a solution to bought reviews (mturk and otherwise) and bots.

Then I realized that it would just incentivise bots to add 1-star reviews to random products once their creators figure out this mechanism.

Sometimes these problems make me sad, it could all be so nice and easy if it weren't for these bad actors.

[BearGoesChirp]

It would be a lot more work, but one could check the validity of ratings based on how other users rate things compared to this user.

Say there are 4 games. Most users who rate all 4 rate them similar, except for the 4th game that always gets really low. So 5,5,5,1 is a normal expected rating, but 1,1,1,5 isn't. So 5,5,4,4 from a high rater or 2,2,2,2 from a low rater would be given more weight than a 1,1,1,5. Other things can be added such as weighting a user's ratings a low impact if they have too few scores to determine ratings from.

This reminds me of the problem of determining the answer key to a multiple choice test given only the answers of the test takers.

[bllguo]

unfortunately sample sizes will probably decrease drastically when you're looking at the subset of people who rated 4 specific games, or other such cases

[fnordian_slip]

Yes, that alone is probably enough to make any system like that come with a huge false-positive rate.

I myself only rarely post a review, and I could easily see how - in a vacuum - my star ratings might look like those of a bot.

[splonk]

This does actually happen, possibly for several other reasons. Bought ratings often come accompanied by other ratings, whether bought or not, just to attempt to cloak the bot. This even happens in systems where there aren't any countermeasures being taken to remove bots yet - I was vaguely involved in spam detection in one and saw all sorts of cloaking behavior even before we'd turned on any sort of mitigation.

[nfriedly]

Yea, that's the main reason why I'm not sure it could be made to work.