Net Searcher Sentiment: An Introduction (Post 1)
Over the next few weeks I'll be posting material adapted from my paper, Net searcher sentiment: A Web-search Based Replacement for Net promoter Score, first published in the journal Applied Marketing Analytics Volume 8, Number 2.
Net Searcher Sentiment (NSS) is a new metric that aims to replace and improve upon Net Promoter Score (NPS).
In Post 1 of this series, we'll start by exploring NPS in more detail, first walking through its benefits, followed by its shortcomings.
From there, in future posts we'll introduce Net Searcher Sentiment (NSS), discuss the methodology behind it, how it addresses the shortcomings of NPS, and the finally look at a few examples of NSS in action.
Introduction
In 2003, Bain consultant Frederick Reichheld published an article in Harvard Business Review arguing that the best predictor of top-line growth can be captured in a single survey question: ‘Would you recommend this company to a friend?’
In the almost twenty years since, Reichheld’s question, referred to as Net Promoter Score (NPS), has become a standard metric for many organizations, complementing and even replacing more comprehensive customer satisfaction surveys. Yet, just as Reichheld argued in 2003 that companies were measuring the wrong thing, NPS itself may be the wrong measure. As we'll discuss, NPS captures self-reported attitudes versus behavior, is prone to sampling bias and runs the risk of non-participation bias.
What is Net Promoter Score (NPS)?
For those not familiar with NPS, it is a survey-based statistic. People are asked on a scale of 0 to 10 ‘How likely is it that you would recommend [brand] to a friend or colleague?’ NPS is then a calculation of the percent of promoters (those who give a score of 9 or 10) minus the percent of detractors (those who give a score of 0 to 6). The resulting value is then multiplied by 100 to provide a score between -100 (every person surveyed is a detractor) and 100 (every person surveyed is a promoter).
NPS = (% Promoters - % Detractors) * 100
Frederick Reichheld famously introduced this metric in a seminal Harvard Business Review article. His goal was to find out if a single metric could accurately predict the growth of a company. Existing metrics at the time focused either on what he called ‘complex black box…customer satisfaction surveys’ or customer retention metrics. He pointed out that these both fall short. In the case of customer retention metrics, while they can help predict profitability, they fail to capture detail on new customer acquisition. He then looked at the current customer satisfaction measurement and determined it fell short as well, providing the example of the American Consumer Satisfaction Index (ACSI) which would have predicted strong growth at companies like Kmart which, instead, slid into bankruptcy.
After work with Enterprise Rent-A-Car, he discovered the potential for relatively simple survey questions to predict customer loyalty and ultimately growth, ultimately settling on the NPS measurement technique as described above. Reichheld discusses how companies found it to be so powerful that in addition to measuring NPS overall, Enterprise’s CEO tied field manager promotions to their branches meeting or exceeding the overall company average NPS.
Shortcomings of NPS
NPS has its own set of detractors. First, since NPS looks at individuals only as promoters or detractors, it misses a tremendous amount of nuance. One person can be both a promoter and detractor at the same time, depending on the context. For example, that person may love a brand overall but have had issues with a specific product from the brand. Depending on how the NPS survey was administered, that person may end up falling into the detractor bucket for specific product issues when in fact they experience a high level of brand loyalty since they love the brand and plan to purchase other products from them in the future.
Additionally, NPS is a survey-based metric as opposed to behavioral based. What people say is often quite different from what they do. Continuing with the previous example, if someone loves a brand but has an issue with a specific product, they may be very likely to buy another product from that brand. By focusing on what they report in the survey, NPS would miss the fact that their behavior differs from what they say.
Third, it runs a high risk of sampling bias. And if bonuses or promotions are based on it, there is a high incentive for individuals to try and weight the measurement in their favor. Imagine a local manager who knows that they will only be promoted if they achieve a certain NPS score. There is a high incentive for that manager to attempt to influence the results by only issuing or recording the survey results for those they are most confident will provide high scores. Many organizations provide local line managers a great deal of autonomy and this may not be noticed for years to come, if ever.
Finally, NPS runs the risk of non-participation bias. Even if everyone is sent the survey to fill out, not everyone will respond to the survey. This leads to the potential that the non-responses skew the data in a way that biases the overall results. For example, people who would rank the product as a 5 may not respond to the survey. In NPS methodology they would have been considered a detractor, therefore under counting the overall number of detractors in the score.
Is there a better metric? The goal is to find a metric with a similar summary capability but looking at points in time instead of individuals. And more importantly, remove the issues inherent with survey-based metrics to instead measure based on behavior. Next week we'll explore Net Searcher Sentiment and how it addresses the shortcomings with NPS. See you then!