Hacker News new | ask | show | jobs
by JasonCEC 3134 days ago
There is actually a whole theory around this, and has lots of implications for the designs of hedonic and sensory scales.

I work on modeling human sensory perception and preference of food and beverage products, and have had to design scales that work as a true "metric";

Most scales suffer from 3 primary problems:

1) avoidance of the endpoints

2) tendency towards the mean

3) minimum information gain

For example; on a 10 point scale, very few (> .5% of respondents) will mark a 1 or 10 (this is problem 1). In addition, 5's are over represented VS the expected amount of 4's and 6's (problem 2).

These problems together reduce the amount of information inferable from the collected data. There is a number of ways to measure this, including information theory (think of the avoidence of the end points and tendency towards the mean as a lossy compression algorithm for the true signal) or as a sampling of an unrepresentative population to infer the posterior distribution.

A 100 point scale has the same problems as above, and in addition suffers from a lack of consistency (reproducibility) - respondents are likely to give a product a different score (say a 92 and 94) when asked about the same product multiple times. This will frequently lead to non-parametric rank reversals, which 1) prove that a 100 point scale is not a "metric" and 2) show that the amount of information is further reduced at higher optionality.

Thus - the discrete scales that work best are:

A) 1 - 7

B) 1 - 13

as they both do not suffer from avoidance of the end points, both have no selectable mid-point (forcing respondents to choose a point above or below the median), and are highly replicable (very few respondents will switch rank orders).

3 comments

One place I worked we changed the scale from 1-5 to qualitative descriptors (I don't remember the exact words but something like "poor, average, good, great, perfect") and it significantly increased the information we got. Previously we almost never got any scores other than 1,4,5. Afterwards we started seeing more 2s and 3s. It seemed that people only needed one value for "bad" so putting "average" at 2 was very helpful.
I must be missing something... why isn't 4 a midpoint of 1-7 and 6 a midpoint of 1-13? I'd think you would want an even number of points to prevent over-selection (like 0-7 or 1-8).
The psychological effect is "tendency towards the mean", and in this case, the mid-point isn't the mean.

Forcing individuals to choose a score above or below the mean yields a better sampling of the true distribution in this case.

What? 1+2+3+4+5+6+7 = 28. 28 / 7 = 4. So 4 is both the midpoint and the mean?
Perhaps people don't differentiate between 1-7 and 0-7 and assume that 3.5 is always going to be the midpoint?
That is correct.
So when they pick 4 on a 1-7 scale, it's not neutral but good, because it's above 3.5?
Perfect, I’m screenshottimg this answer and showing it to my friends when they get weird when I ask them how much they like their beer on a scale of 1-7.

Because science.