| Well, let's let the author try to convince you that NPS is a harmful, horrible number to summarize a company's performance on. He would tell you that NPS is only like earnings or revenues if we allowed either to have 50% or more of their data filled with arbitrary numbers, not audited data collected from state-licensed specialists who would lose their job if it was discovered the data was manufactured whole cloth. The author would also tell you that NPS is easily gamed and there's no checking on whether that is or not. He wrote extensively in the article the various techniques that folks can game the numbers. If this is a number reported to shareholders, shareholders should insist (No, Demand!) that the numbers be corroborated by a neutral third party that will accept liability for any errors. (No surety insurer will guarantee such a liability, for the risk of error or misrepresentation is way too high.) As you stated, most use follow-up questions to get a richer understanding of the customer. What the author would tell you is that it's clear the NPS recommendation question taints those followup questions and diminishes their validity and inherent value. If the true goal is to learn a richer understanding of customer experience, there are many better ways to achieve it. In other words, the author believes if executives want a simple metric that is better than NPS, a random number generator is the fastest and cheapest way to achieve it. Why bother with customers at all, if all you're going to do is squander your interaction with them on such a foolish metric. — The author. |
These types of sweeping statements aren't a helpful way for the author to advocate his broader point.
The strengths of the article lie in the more detailed points which bring to light some great examples of how NPS, and surveys in general are mis-used (and some examples that don't actually manifest themselves as real problems in daily use very often).
My thoughts on the examples chosen above:
>50% or more of their data filled with arbitrary numbers
I'm not sure what this refers to, but the way the NPS equation works, every respondent's score matters and mathematically impacts the overall NPS (each one feeds in as either a promoter, neutral, or detractor). Some of the author's own recommended questions include only 3 answer inputs also.
>Not collected from state-licensed specialists
Almost all operational data collected by companies for management reporting is not collected by state-licensed specialists, but is still useful.
>Easily gamed
All survey questions could be gamed in the ways similar to the article examples. A good executive will make sure the survey is asked in the same way of his own organization as of those of peers, and in the same manner over time. He or she won't let agents do things like cherry pick which customers to survey, otherwise the money he invests in the survey wont' actually help him run his company.
>Shareholders should insist the numbers be corroborated by a neutral third party
Correct, companies who are serious about measuring NPS accurately do often hire third parties to run surveys (see example of JD Power running NPS: http://www.jdpower.com/press-releases/bain-certified-net-pro...)
>NPS recommendation question taints followup questions
All survey questions can be tainted by preceding questions. When writing a survey it is fairly straightforward to A/B test the order to make sure this isn't a major factor.