When trying to gauge people’s perceptions and experiences of a financial product or service, few questions are as straightforward as, “Would you recommend this to a friend?” The question can be a simple, qualitative counterpoint to more complex models for understanding the value products have for low-income clients such as Client Math, Financial Diaries, or studies that aim to measure impact.
It gives customers themselves—rather than their financial circumstances—a say in the utility of a product. For these reasons, we have commonly included this question in surveys of various pro-poor financial products. Traditional marketing literature often considers this, ‘Net Promoter Score,’ an industry standard for measuring brand loyalty, which is purported to be the single most important indicator of future earnings for a company. Other researchers disagree, having failed to find statistically-significant links between the two.
Over the course of our own research, we’ve noticed problems endemic to this question. We often conducted surveys in which a customer expressed disappointment in every aspect of a product only to conclude the interview with the answer that they would, in fact, recommend the product. Such surveys are not aberrations. Instead, we commonly find hard-to-explain mismatches between products’ client value and the apparent willingness of their users to promote them.
In markets where populations are typically excluded from access to services, we are finding that Net Promoter Scores might be an insufficient measure of pro-poor financial services’ value proposition and even of respondents’ other subjective experiences.
There may be a few reasons for this:
(1) Asking the question in a close-ended way may be ill-suited to getting at nuanced feelings around loyalty to a product or service, particularly if it is the only available option. This effect will be especially strong in developing countries, where the number of options of financial services is often very limited.
(2) Beginning a question with “Would you…” may inherently lead respondents, especially those less-engaged in the survey, towards “Yes.”
(3) The strong identification that low-income clients have with their distribution channels often obscures the specific understanding or critique of one financial product. A good recommendation may be more reflective of trust in the channel and good service rather that the use a product was able to provide.
(4) Most significantly, we worry that respondents may feel comfortable complaining about a product on their own terms, but not want to admit that they’ve purchased a ‘dud’ when asked pointedly. It’s natural for people to be biased towards things they’ve purchased. This effect may be stronger among low-income people, previously excluded from financial services and with less money to spare.
Our experiences with willingness-to-promote questions and the skepticism around the Net Promoter Score in academic research have convinced us to rely less on this measure. It seems implausible to ask respondents to put themselves in the role of a promoter of products not discussed in normal conversation. This role may reflect social considerations more than actual experiences using a product. Given the extent to which we’ve seen these biases in our findings, we no longer ask the question. Giving respondents a space to reflect their feelings about a product is worthwhile, but in our world. It may require a more carefully-designed questions to realistically unearth.