[invalidfunction]

Eh - humans generally have a hard time rating things on a 5 star rating system. That's mainly because the discrete intervals between 1 and 5 stars aren't well defined so everyone will have a different opinion of what 3 stars "mean". I encountered this when I ran a rating system in Mechanical Turk. If you ask turkers to do something arbitrary like "rate the quality of this picture" it takes quite a number of ratings on the same image to get an accurate consensus.

It would be interesting to see if you can replicate experiment but replace the 5 stars with 5 yes/no questions regarding their performance and see if those match up

[dogma1138]

Whats worse is that any type of rating system has an utter bias towards the very very upper end of the scale by it's consumers.

If you have a true rating system then 2.5/3 out of 5 means average or more or less what you should be looking for in most cases 4 is out standing and 5 is bloody icecream pooping unicorn.

You see this every where a rating system is implemented heck in entertainment movies and games don't fight from 0 to 100 anymore but form the 90-100 if you see any score below 90 or 4 stars people usually avoid that product.

[pc86]

There are text descriptions under the stars, which are coincidentally rated out of 4, not 5. 3/4 is "Solid" for someone's technical skills. If someone does below average or amazing it's unlikely that the interviewer is going to select 3/4, regardless of their internal idea of what 3/4 means, when it says "Solid" underneath that.

[saulrh]

Even worse because 1 star and 5 stars mean different things to different people, or even sometimes to the same person under different circumstances. If you asked me to rate my interview performance without first defining some objective criteria, for example, I'd have to spent a few minutes deciding whether (1 vs 5) stars is ("didn't get the job" vs "got hired with a nice title and negotiated a bunch of extra pay") or ("was arrested for assaulting the interviewer" vs "got the job").

[zyxley]

I'm a fan of the Siskel & Ebert system instead of five stars: two thumbs down (bad), one thumb up/one thumb down (okay), two thumbs up (good).

[InclinedPlane]

Right, it's like game ratings. Is the minimum rating an "F" or is 60% an "F"? Is a rating in the middle good or bad?