Why is research so risk averse?

This post started life as a response to Ellen Woods’ well thought out Greenbook post on the paradox of risk, which you can read here. However, here is an extended version of my thoughts on the difference between the herd and good for the individual, in terms of risk, risk avoidance, and its implications for marketers and market researchers.

In this post I want to concentrate of two issues: 1) the difference between what is good for the economy and the individual, and, 2) the difference between the short-term and the long-term.

The difference between what is good for an economy and what is good for an individual.
Most innovations fail, most entrepreneurs fail, most new products fail – the failure rate is typically quoted as being in the 80% to 90% range. Given these high failure rates, the logic for each individual entrepreneur is that they should not risk everything on some new change, idea, or innovation. However, the benefits to the economy of the few innovators and entrepreneurs that succeed is massive. Indeed, I think the number of entrepreneurs in the US is one of the keys to its success over the last 100 years – a success which is the result of a massive number of failures, producing a large number of successes. However, this difference between the individual losses and the collective gain raises an issue for market research. If we are accurate in our work, we should be telling most of our clients that the chances of success are well below 50% – which means research will typically and most often be a drag on innovation, something which inhibits change. (I am not saying research always inhibits change, but, probabilistically, it will inhibit it more often than it promotes it.)

What should research do about its dismal science of predicting probable failure? Research should not deceive; it should not tell people who are going to probably fail that they will succeed. Research can help them minimise risks of failure, but, in many cases, risk minimising strategies also limit the potential upside. This is one of the paradoxes. We want everybody to research their ideas, but if they do, we would probably reduce the rate of growth and development in the economy. Clients need to assess their risk profile and fit the research to their needs.

The difference between what is best in the short term and what is best in the long term.
In the short term, for most profitable companies, the best financial return for the current year comes from not changing anything, i.e. business as usual. And, in a world of short-termism, the impact of the latest share figures and this year’s performance can be massive. However, as Ellen points out in her post, companies that do not change fail, in the long run (and the long run can come quite quickly as Kodak found out). One consequence of this paradox is that it is the failing companies that often adopt the most innovative and risk-taking approaches. For example, when P&G first appointed AG Lafley as their CEO they had just lost $85 billion of market capitalisation – this gave Lafley the permission and the freedom to change P&G in fundamental ways (as he recounts in his book The Game Changer).

When I look at the issues surrounding insight, consultancy, and market research I am reminded of the old joke about “How many psychiatrists does it take to change a light bulb? One, but the light bulb has got to want to change!” Can market research help foster innovation, change, and growth? Yes, but companies have to want to change. For example, if major clients were to decide to stop all the brand trackers, customer satisfaction, and NPS type projects on Jan 1st 2014, the research industry could have better, cheaper, more appropriate options in place. But, my prediction is that it will be the new companies who don’t know the old ways, and the failing companies, who feel they have to roll the dice, who will drive the changes. The bulk of the big spenders on market research will follow some years behind.