Predicting the future for future predictions

The World has just seen the result of what happens when people mistake the purpose of market research. Arguably the highest profile survey has just been completed and it has done nothing for the credibility of the industry that I work in.

The EU referendum (or Brexit) was essentially a survey asked to the whole of the UK. It was seemingly a simple question, ‘Should the United Kingdom remain a member of the European Union or leave the European Union?’ The result of this survey is notable not only because of the potential ramifications of the results – but also because the polls (final predictions from research just before the result was counted) got it wrong. Not dramatically wrong (they were in fact only 4-6% away from the actual result depending on the source, but this is outside of the margin of error) but the final polls all had Remain winning. This became the news. This potentially impacted the financial markets and influenced voter’s decisions on the day of the referendum itself. This was a situation where ‘prediction research’ was taken as gospel. This was wrong.

The trap that people can fall into is expecting research to provide all the answers. Research is not a crystal ball, but it is a powerful practice that if understood, and used correctly, can help provide companies with a competitive advantage.

 In order to help remove any doubt – I feel that we need to start referring to research as 3 completely separate disciplines: predictions, measurements and inspiration. And for each, be clear in the amount of robustness needed from each, and be clear how confident we should be in treating the results as fact

Predictions
This is perhaps the area that most people misuse. There are many new methodologies that have come to market in the last decade – all aimed at more effectively understanding customers. From shopper journey to concept testing, we are looking to understand if our product or service or idea will be accepted. The best methods place the actual product, in the arms of the customer, in a real world setting, for a number of weeks, months or years. Pilot schemes, trial runs and placements help build up a strong body of evidence. The point here is that this research is thorough and exhaustive, asking multiple questions in multiple formats. We must be wary of ‘cheap and quick’ research that promises to do all this through a 5min survey (political polls take note!). When done correctly the robustness of predictive research can be strong, but it can never be perfect. Again a watch out for any future referendum polling.

Measurements
This is the classic, Usage and Awareness study, NPS or benchmarking project. Companies want to understand where they sit vs their competitors. This is useful information. However, we again have to be wary about who we are asking and in what context. Have you heard of brand X? Yes. Would we recommend brand X? Yes. Would we be likely to buy brand X in the future? Yes. These are useful metrics to track over time. However, they are less useful if there is no understanding of why? Crucially, what is driving this attitude and how can a challenger brand change perceptions and steal share? Measurements are not effective if we don’t understand what we can do to leverage brand X’s position. The robustness of measurements comes not in their predictive power, but in their trends – what we can see is happening over time. Here then again, measurements are not perfect, but provide a guide on how the market is changing – highlighting when there is need to dig deeper in the why of what is happening.

Inspiration
Arguably this is the most powerful form of research. But it is also the least robust. Immersing yourself in customer issues and using research to help generate new ideas. The sort of ideas that research should develop can help incrementally improve competitive advantage. Which, is then measured and explained and then we look for new ideas. Over time, these incremental improvements add up, until you have a stronger offering, that is closely aligned with customer’s needs.

We need to stop thinking of research as something that we can guarantee will give you a rock, solid robust answer. And instead see it as a guide, the validity of which depends on the amount of time, effort and respect you afford it. After all, you can’t put a price on a great idea.