[nulbyte]

I would go a step further: NPS is a garbage metric.

First, you start by assuming your customers can even reasonably ascertain their likelihood to recommend. They can't; there are people who answer 10 but will never recommend, and there are people who answer 0 but already have and will again.

Next, you assume your customers are idiots and don't know how an 11-point scale works by adjusting the midpoint: Instead of 5, the middle is now 7 and 8.

Then you realize there are two many numbers, so you throw several out by reducing your 11-point scale to a 3-point scale, after which you re-interpret "unlikely to recommend" as "likely to snag some other customers on my way out the door."

Finally you calculate your 'net promoters' by subtracting the percentage of low scores from the percentage of high scores to give you a nice round number that doesn't correlate with what's actually happening in the real world.

And this is just what happens when you do it 'the right way.'

NPS is said to measure growth using loyalty as a proxy. But then, what does that have to do with recommendations? Nothing.

[gyomu]

Bahaha oh man this brings back memories. I worked for a startup where the CEO spent weeks working on determining our NPS and talking our ears off about it. At the end of the process, he was so happy - our NPS was higher than Apple! I didn’t really know what that meant but I figured he had an MBA and knew what he was doing.

The startup never became profitable and ran out of investor money 18 months later.

There were many things wrong with the company but this was one of the things that made me most feel like I was in Office Space.

[timkam]

Based on practical experience, NPS is garbage because:

1) Even with stable mean and median, NPS tends to vary month over month, at least for my B2B settings where samples are probably much smaller than for B2C. Then, management goes nuts because of very subtle shifts in the distribution caused by NPS' arbitrary aggregation into promoters, neutrals, detractors. Of course, often investors are married to NPS, so educating management does not solve the problem.

2) NPS varies unreasonably across cultures. We used to say, somewhat tongue-in-cheek, that NPS is a US-centric metric, where things are either amazing or awful (with little space in between). E.g., in northern/central Europe, an 8 can be pretty amazing.

[kmoser]

NPS is the electronic version of a poorly run focus group. People will assume all sorts of things and bring in all manner of bias. You may get some data, but you have no idea how accurate it is. Even worse, you have no idea why they answered what they did, or how to fix things to make your customers happier.

[pflenker]

Not defending NPS here - I don't use it - but some of your assumptions are wrong.

> They can't; there are people who answer 10 but will never recommend, and there are people who answer 0 but already have and will again.

What matters with NPS is trend over time, and getting the numbers at a scale. Yes, there are people who randomly click on one end of the scale or the other, but the assumption is that on average the portion of these people is stable.

> Next, you assume your customers are idiots and don't know how an 11-point scale works by adjusting the midpoint: Instead of 5, the middle is now 7 and 8.

This does not come from the assumption customers are idiots, it comes from the idea to treat people who vote "in the middle" not as neutral, but as detractors. Which makes sense: If someone tells me "hey I know Product X and it's meh", then I'm less a promoter but more a detractor.

> Then you realize there are two many numbers, so you throw several out by reducing your 11-point scale to a 3-point scale

The 3 point scale was the goal all along though, it's the idea of an asymmetric scale that leads to the 11-scale to 13-scale reduction.

> after which you re-interpret "unlikely to recommend" as "likely to snag some other customers on my way out the door."

If your assumption is that promoters drive positive growth, it's fair to assume that detractors drive negative growth by recommending an alternative. If you believe in that core assumption that NPS measures word of mouth, then this interpretation of "likely to snag some other customers on my way out the door" is a sensible one.

> NPS is said to measure growth using loyalty as a proxy. But then, what does that have to do with recommendations? Nothing. I don't think the underlying assumption is bad. That's how influencers work: people are more likely to buy something that is being recommended to them by someone they trust and someone who is passionate about the product.

Does NPS work? I don't know - I'm not using it as I said above. But at least the assumptions under which NPS are designed on top of the idea of word-of-mouth as a growth diver seem solid to me.

[nulbyte]

> Yes, there are people who randomly click on one end of the scale or the other, but the assumption is that on average the portion of these people is stable.

That assumption is predicated first on the idea that people can and will tell you their likelihood to recommend within a reasonable degree of accuracy. I don't think they do.

1: https://www.xminstitute.com/data-snippets/gap-consumer-recom... 2: https://hbr.org/2019/10/where-net-promoter-score-goes-wrong

> If you believe in that core assumption that NPS measures word of mouth, then this interpretation of "likely to snag some other customers on my way out the door" is a sensible one.

But that's not the scale given to the respondent. The scale is given as going from "not at all likely" to "very likely" to recommend. There isn't an option for likely to recommend against. The low end of the scale probably captures some, but to assume it is near 100% is a mistake.

[pflenker]

Re. your first point: When working with NPS, you would not actually assume that 1.000 people who voted with the highest possible value actually translate to 1.000 actual recommendations. I think not even the most vocal proponents of NPS would claim this. What matters is the _trend over time_, and that trend allows drawing conclusions regardless of actual individual customer behaviour, unless you have reason to believe that what customers actually do is completely independent of the score they give. If today 50% of respondents rate positively, and tomorrow 25% do, for large enough pools of respondents, something has gotten worse and it's not a stretch to assume that the total number of recommendations that go around for your product will go down.

Re. your second point: You are right that it is not explicitly asked for, but that does not imply that the core assumption would be faulty.

So much depends on the actual product being sold, and many other aspects, but being an imperfect metric or only a rough approximation would not imply that the metric itself is garbage, as the comment I replied to stated.

Here's a simplified example: Let's say I own a web shop, where growth implies a growth in sales. Someone on the positive end of the scale is more likely to shop again (contributing to keeping the current growth rate stable), and more likely to recommend my shop to others (contributing to positive growth). Someone in the middle or on the negative side is less likely to shop again (contributing to negative growth) or even actively recommend against my product (counteracting any positive growth the promoters would cause).

Does the NPS tell me anything about _actual additional sales_ I can expect? No. Does it tell me anything about _actual customers I will lose_? Also no. But it is one predictor of future growth, and as such useful.

Does my speedometer tell me if I'm driving in the right direction? No. Does it show how many traffic jams are ahead? Also no. But it is one predictor of when I will arrive at my destination, and as such useful.

[naijaboiler]

Thank you for taking time to explain this to people who are think NPS should be interpreted literally.

No, it’s really just a decent enough proxy for how your product is trending over time among your users.

Individual responses are uninterpetable. Single NPS survey is also not very interpretable. But with enough people and over enough time, it is a useful signal