Image of Survey

Should Market Research Still Be Using Significance Testing?

Over the last few years there have been many calls for market researchers to stop using significance testing based on assumptions of random probability testing to measure the potential impact of sampling error. For example, Annie Pettit writing in The Huffington Post asked “Stop Asking for Margin of Error in Polling Research”. But, despite the concerns about the correctness of using this technique, it seems to still be in common use. In this post, I briefly explain what significance testing is (experts can jump this bit), why it doesn’t do what people seem to think it should do, and the way I think we should be using it in the future. What Is Significance Testing? The type of testing I am talking about in this post relates to sampling error. In quantitative research, a sample is taken from a population and one or more statistics are calculated. These statistics are then used to estimate the values for the total population. For example, assume 1000 people are selected at random from a population of 20 million. Assume that 50% of the sample are female. The inference from this study is that it would be expected that 50% of the total population […]

Photo of Lego

You Can Combine Standardisation with Creativity

One of the key tools for increasing productivity, reducing errors, and facilitating automation is standardisation. Unfortunately, many people think there is an association between standardisation and a lack of creativity. However, standardisation does not need to create a cookie-cutter approach, indeed it can promote creativity when done the right way. Whitworth Standard – 1841 For most of the Industrial Revolution, every blacksmith and workshop used its own dimensions for nuts and bolts. If you bought a steam engine from one workshop, you had to go back there if you needed any work doing on your machine. This slowed down the adoption of new machines and tools. In 1841, British engineer Joseph Whitworth introduced the British Standard Whitworth system, the first standardised system for engineering in the modern world. This innovation led to changes in how things were manufactured, for example it was now easy to buy parts from several companies, it was easier to scale-up production, which led to a golden age in manufacturing and industrial creativity. Avoiding Procustes The worst type of standardisation is often referred to as cookie-cutter, or more scathingly as procrustean (definition: enforcing uniformity or conformity without regard to natural variation or individuality). Procustes was a […]

Social Media Banner

Social Media in 2017, Not much change, but lots of progress

Last week I attended the MRS Social Media Summit in London, and was struck by something that at first seems to be a contradiction – not much change over the last seven years, but lots of progress. Not much change! Back in 2010, I published The Handbook of Online and Social Media Research. Nothing presented at the Summit was fundamentally different from the picture in 2010, in terms of the aims of the tools, the range of approaches adopted etc. Lots of progress! Compared with 2010, researchers and research users are much better at using social media in a research context, in particular using blended techniques that use social media in conjunction with other approaches. Social media hasn’t changed, it has simply got better. Key Themes from the Summit The one day MRS Summit, well curated and chaired by Marc Brenner, provided several interesting themes and lessons for research providers and users alike, including: Blended research is often the key to success Social media is more than just Twitter WOM is more than just Social Media The human element is still essential Blended Research The main theme of the day was the benefit of blending social media research with other […]

Discovery: a new tool for accessing longitudinal cohort studies

Are you interested in longitudinal data? For example, The Hertfordshire Cohort Study (following 3000 men and women since they were born in the 1930s), or the British Cohort Study (17,000 people born in a single week in 1970). If so, you will be aware of the problems such as: knowing what data is available, what questions were asked, and where is the data stored. Discovery is a new tool (still in a relatively early stage of development) from CLOSER, that makes eight longitudinal studies more accessible. Please try it, please leave your feedback. Where are the variables I am interested in? Perhaps the best way to understand the usefulness and power of Discovery is an example based on finding variables of interest. The image below shows the home page of Discovery, and it shows that the eight studies include over 55,000 variables [note, if you click on the images they should enlarge]. With so many variables, nobody wants to scroll through them all to find items that are useful to their project or query. By clicking on Variables we can filter the list by Study, Life Stage and Topic, as in the image below. In this example, let’s assume we are […]

Rooster Happy New Year

Asia Pacific Predictions for 2017

The APAC (Asia Pacific) region’s two New Years are relatively close together this year, Western New Year on January 1 and Chinese New Year on January 28. So, this seems like a good moment to ask various people from the region to tell us what they think will be the key issues in APAC in 2017. Here are nine predictions focused on APAC, but embracing the world. Becki Southern Marketing Manager Asia Pacific, Lightspeed, Australia Automation will undeniably present opportunity to keep consumer data and understanding top of mind but, with systems replacing human decision making for marketers, Market Research will need to ensure its relevance and position within the planning process. Focus on creating stronger partnerships with clients will allow researchers to better understand this shift and act accordingly. This will mean acting differently, developing new ways of working and new solutions that fill the gaps for the 2017 marketer. This comes down to having the right technology to add to the consumer data brands already own. Harnessing this to complement strategy and build a deeper understanding is imperative to keep focus on agencies and suppliers outside the realm of their own data. As younger markets in APAC grow, […]

Image of Hong Kong

What can the Asia Pacific Region teach the MR world?

I think some of the best thinking about new market research comes from the Asia Pacific region and I want to share five examples that are helping re-shape the way we envisage and do market research. All five examples will be presented at the upcoming, all virtual, online Festival of NewMR – sharing ideas from Japan, Australia, India and both mainland and Hong Kong China. Shobha Prasad, Drshti Strategic Research Services, India The Fickle Mistress: Loyal consumers changing brands and the change-constancy conflict. Shobha highlights the impact of brand renovation on loyal consumers, and the role of the Change-Constancy Conflict in the loyal consumer’s response to such changes. By analysing multiple cases over the last decade Shobha has determined the stages and allied emotions that the consumer goes through, and has assessed how this plays out across different categories and consumer types. Sign up for our APAC Tuesday 28 February Webinar by clicking here. Mike Sherman, Marketing, Customer Insight & CRM/Big Consumer Data Expert, Hong Kong Big Consumer Data: the Promise, the Overpromise, the Opportunity Mike tackles the main criticism of Big Data, that it is all talk and no action. By looking at Big Data successes and failures Mike […]

Future Decisions image

Most Predictions Are Wrong, So Let’s Keep Making Them

The researcher and author Philip Tetlock has been researching predictions since the 1980s and has concluded that most predictions fail, that most experts do worse than a chimp with pin (i.e. worse than chance), and that predictions more than a few months out are particularly flaky. If you are interested in the topic of predicting and forecasting then I strongly recommend his book Superforecasting: The Art and Science of Prediction, which illustrates the problems and shows why and how some people do better than experts. But despite the dismal record of most forecasters and predictors, we continue to make predictions, especially at this time of the year. Indeed, I recently published my “Five Market Research Trends for 2017” and was pleased to see that over 4000 people had read it. Why Predictions Can Be Useful I firmly believe that making predictions, and urging other people to make predictions, can be a really useful exercise, despite the poor track record that predictions have. In order to assess the ways predictions can be useful, we need to separate the different categories and motivations for forecasts and predictions. So, here are seven categories/motivations – and the benefits they can bring. 1 The Planner […]

Blue sky and parachute

How did my predictions for 2016 fare?

I have just published Five Market Research Trends for 2017, covering issues that I think will be important over the next year. So, it seemed only fair to look back to this time last year to see how my previous predictions performed. A year ago I published “My Predictions for 2016”, which contained nine predictions. Here is an assessment of how I think those predictions performed. 1) Bigger Legal Problems for Facebook and Google. This has been true in the sense that they have faced a growing number of court cases, especially in Europe, along with investigations into their poor records in terms of paying tax. However, the problems they have faced have not markedly impacted their ability to grow or make money. 2) Automation. We saw lots of movement on Automation in 2016 and that trend looks likely to continue into 2017 (check out the NewMR Debate on Automation being held as part of the Festival of NewMR). 3) Surveys will continue to suffer. This is a slightly tricky one to judge. Each year for the last few years (according to the EOSMAR Global Market Research Report) surveys have become a smaller part of the total research mix, but […]

Koi banners

Five Market Research Trends for 2017

At this time of year our thoughts turn to what will excite and challenge us in 2017. Here are my five top picks. Automation This has been one of the key trends for the last couple of years, although much of the action has been out of sight; taking place in the operational backrooms of clients and agencies. Earlier this year, Lenny Murphy and I published a report on Automation, and I see little reason to change our view on the scale and direction of change. Automation is going to underpin most success stories over the next few years. At its best it will reduce costs, increase speed, and provide an opportunity to provide more evidence-based decision making. In some cases automation will result in a less good product, but in may cases the standardisation will enhance quality, as will the ability to provide relevant answers at the speed and cost needed and the ability to scale things up. Insight Finding Data is getting cheaper and more abundant, but techniques for finding the story in the data have barely moved forward in the last thirty years. That is beginning to change, as evidenced by the popularity of our series on […]

future start icon

Want to know more about Behavioural Economics?

This week, NewMR broadcast four webinars on Behavioural Economics (BE) – as part of our Research Innovation event. If you want to expand your knowledge of the theory and application of BE then you might find these presentations useful. Stephen Paton, Behavioral Economics – Is that a wolf at the door or opportunity knocking? In this 16 video Stephen give a great overview and introduction to BE and where it fits in the research picture. Leigh Caldwell & Lizzi Seear, Behavioural economics gets real: with behavioural conjoint In this 21 minute video Leigh and Lizzi start by making the case for implicit measurements and then go on to show, via a Holiday Inn case study, how conjoint analysis can be updated using behavioural economics. Colin Ho, The Connection Between Big Data and Behavioral Economics In this 14 minute video Colin shows how BE can be used as a lens when working with Big Data. Raj Sandhu, Behavioural Economics: Marketing’s latest shiny new thing, or a tipping point? In this 17 minute video Raj highlights the need to evaluate implicit measures and goes on to show how BE can be applied to the development and evaluation of snack foods. Note, the […]