What is the best market research approach?

Who What Why and WhenPosted by Ray Poynter, 6 October 2013

I quite often hear somebody say that X is the best research approach, where X might be eye-tracking, ethnography, behavioural economics, discrete choice models, nano surveys, or any one of twenty other contenders. However, any answer that starts with an approach is, in my opinion, wrong.

The best market research approach starts by looking at a specific research question and then trades-off three elements, quality, speed, and cost – typically by trying to find something that is good enough, fast enough, and cheap enough. Assessing the speed and the cost of an approach is normally straightforward. In terms of cost, if everything else is equal, the lowest price is best. In terms of speed, there are speeds that are too slow, speeds which are OK, and sometimes a point when faster adds no additional value.

Quality is based on supplying something which meets the needs of the client, and it is this element that guides the researcher to determine the best approach, i.e. to recommend the cheapest/fastest solution that provides what is needed.

The seven questions below suggest a possible hierarchy in assessing what is likely to be the best research approach in a given situation. If level 1 answers the research question, it is likely to be best answer, i.e. the best trade-off of quality, speed, and cost.

1. Does the answer (data) already exist? All too often research is conducted when the answer is already sitting on a shelf (although these days the shelf is typically virtual).

2. Can we just ask people? For many research problems, a simple question is the best way of finding out an answer. What is your address? What type of car do you drive? Do you use Facebook? In most of these cases, subject to issues like social desirability bias, simple questions, asked via a survey or form, work well.

3. Do we need to quantify it? If we want to know what people think an ad means or whether they understand how to use a website, then a qualitative piece of research, for example focus groups, is quick, easy, and effective. If the research questions are relatively simple, an online focus group is likely to be sufficient.

4. Can we ask people questions and model the results? Asking people which party they are going to vote for or whether they will buy this new type of breakfast cereal leads to answers that do not directly relate to what people do (partly because of bias, partly because people don’t know what they will do), but in many cases the answers can be modelled, weighted, or compared with benchmarks to give guidance on the likely outcome, and about the probability of that outcome.

5. Can we modify the questioning to get people to reveal their inner motivations? If people can’t tell us how they make decisions, and if their answers to simple questions are not useful for modelling, then the research can be extended. For example, in qualitative research projective techniques can be used, in quant we can use tools such as choice experiments or prediction markets. There are a growing number of ways of modifying the questioning, for example virtual environments, implicit association, gamification, and other techniques including neuroscience and facial coding.

6. Do we need real life observations? If we can’t find out the answers in a laboratory setting (such as a survey, a central location test, a focus group, or a depth interview), then observations need to be gathered from real life, for example by asking people to collect slices of their own life via a smartphone, or by recruiting people to visit homes, workplaces etc to gather information.

7. Do we need ethnographers, anthropologists, ethnomethodologists etc? If collecting data about people’s everyday lives is not enough, then the next level up (in terms of taking time and spending money) is sending trained researchers into real life situations, to seek out the clues, to interact with people, and to find the hard to reach answers. For example, to really understand kitchen hygiene practices, perhaps to find out about gaps in processes, questions and observation are unlikely to be enough, researchers will need to be there, and be there long enough for people’s behaviour to return to normal. Ethnomethodology, for example, might employ breaching to gain a deeper understanding, perhaps by wiping the bench with the hand towel and watching what happens.

These seven levels do not list every approach, but other approaches can be assessed against these seven to see if they provide a faster, better, or cheaper solution. For example if a specific question can be answered by social media monitoring it is likely to be at the faster/cheaper end of the spectrum. By contrast, even if semiotics can answer a particular research problem, it is unlikely to be very cheap or fast, so it tends to only be used when cheaper and/or faster methods can’t deliver the results needed.

It should be noted that the answer to a market research question does not have to be market research. If there is something else which provides a better solution to the three-way trade-off of quality, speed, and cost, then that is the best solution. For example, when websites first burst on the scene, the best way to find out the basics of who is visiting was via market research, often using pop-up surveys. However, the market developed and the ‘best’ solution for many situations was provided by analytics. Market research has no automatic right to exist, market research should only be used when it is the best solution. For example, many online service providers are finding that A/B testing is a faster/cheaper/better way of testing service and offer variations, making research redundant in some cases.

If you want to know more about answers to contemporary market research questions, check out ESOMAR’s new book, edited by NewMR’s Sue York and Ray Poynter.