In market research, is agnosticism the new belief?

Quoit

Traditionally the term agnostic has been applied to people who have not had the courage of their convictions to settle for belief or refutation. However, over the last few years the term agnostic has become increasingly used in the area of market research, and indeed agnosticism appears to be the new creed for many of the suppliers to our industry and the prediction for many of the pundits forecasting the future.

Among the key areas where agnosticism is becoming a driving principle are:

  • Mode agnosticism – especially between online, mobile phone, and tablet.
  • Pull-Push agnosticism – between apps and browser based mobile research.
  • User agnosticism – between DIY, assisted serve, partners, and full-service.
  • Code agnosticism – between classic market research conducted under research codes, and other forms of research, such as Big Data and Social Media Research, which are as likely to be offered by non-research companies as research companies.
  • Sampling agnosticism – market researchers used to be believers in ‘the way’ (aka random probability sampling), but now the largest single method is the convenience sample (aka online panel) and alternatives are picked according to their merit (especially availability, speed, and price), rather than on a priori beliefs.

Why the change?
To me it seems as though there are three main reasons for this shift to agnosticism:

  1. The least good reason is that there are growing numbers of people in the research industry, as buyers and sellers, who do not know the beliefs (why one way might be methodologically better than another), so their agnosticism is based on not knowing and/or not caring.
  2. The second is a recognition that many of the most important questions should be answered by the buyer, not the seller. In the market research world before 2000 it was not uncommon for researchers to assert that they knew best, that they should define what was right and what was wrong, and to create the standards for what clients would be allowed to buy. If vendors realise that buyers are the people who should be defining standards and making choices, then they realise that the vendors need to be more agnostic, and less belief driven.
  3. It is currently very, very hard to know what is going to happen, over the next few years, with issues such as mobile versus online, with DIY versus full-service, and apps versus browsers. Some small companies will probably want to bet their future on a particular outcome, but most organisations will want to hedge one option against another.

So, what are your views? Do you agree that agnosticism is growing? If so, is it a good thing or a bad thing?

2 thoughts on “In market research, is agnosticism the new belief?”

  1. I’m not sure I fully understand your article – it seems to conflate some different points. On one hand you argue that agnosticism is based on poor understanding of methodologies. On the other, you seem to be suggesting that market researchers won’t commit in terms of predicting the future. However, there is an emerging rationality in terms of ‘future prediction’ – largely as a result of ‘radical empiricists’ like Nassim Taleb. He asserts (and he’s right) that anyone who suggests they can predict the future (and we have to include market researchers who use research data to create forecasts) is a fool and/or charlatan. He’s right. Future prediction is a mug’s game. Therefore researchers are right if they hedge their bets – and don’t over-claim to what extent data compiled from lots of people with opinions might help predict the future. To some extent this might help explain the methodology ‘agnosticism’. There never is a right or wrong answer in terms of methodology when market research really is such an imprecise “science”. In fact it’s not a science at all. Give a bunch of researchers the same data set and each would write different reports with differing conclusions. So, if the reporting is so subjective what does it matter what methodology has been adopted?

  2. Interesting post Ray. I like to think of myself as ‘pragmatic with standards’ which you could call agnostic.

    Pragmatism with standards is the heart of any good quality hybrid research. As an example, in a current project we interviewed part of the sample online and part through CATI – simply because that method was best for the respondent – and then combined the results. This is probably a fairly common thing to do, but years ago, I would have taken the purist line and let the methodology dictate to me what to do, and kept the results separate and therefore less usable. The point is though that we didn’t do this blindly; we compared the results from each method before we combined them to make sure they weren’t different in ways that could not be explained by other factors.

    I think the rise in agnosticism is a good thing, if I interpret that to mean the ability to consider different points of view rather than blind adherence to an established approach. I personally think that buyers are better off using suppliers who are adept in multiple methods, rather than those who push just one. That’s the best kind of agnosticism.

    There are two kinds of people who push just one – the lazy and the purists!

    1. There are some lazy suppliers who will use the cheapest/easiest method without giving the consequences any thought. (It is the supplier’s job to do this not the buyers).
    2. The MR industry has also had its fair share of ‘I am right’ purists for whom the only way is their way.

    Sue

Comments are closed.