MR Agencies and the Service they give Clients

I have just finished listening to a great presentation by Suz Allen (R&D Director Sensory & Consumer Science Asia Pacific & International, Campbell Arnott’s) talking about how suppliers and clients can work together better (you can access the recording and slides here). Whilst I found the presentation useful, informative, and entertaining, I was amazed at how low the bar seems to be. I think it is distressing that agencies are making such basic mistakes. Here are some of the recommendations that Suz made: “No Surprises! Never!” For presentations, arrive early, ask (in advance) if you can have early access to the room to set-up, have spare cables, connectors, clicker, etc (we should not need to be reminded of this!) Match your staff to the client, some people work better together than others, this is a people business. Call your client, 1, 3, or even more months after a project to ask how it is going. The agency should seek to make the client look good, their “butt is on the line” when they hire us. Value and reward good clients, for example sharing ideas, papers, leads, and recommendations with them. Suz Allen’s presentation has lots more tips on best practice, […]

Short-term effects do not predict long-term advertising success

“Half the money I spend on advertising is wasted; the trouble is I don’t know which half.” is one of the most commonly quoted comments about advertising, being variously attributed to John Wanamaker and William Lever. Perhaps as a consequence, one of the key uses of market research is to test, monitor, and track advertising. However, it might well be that half of the money spent on testing and tracking advertising is also wasted. How does advertising work? In the distant past we used to think advertising worked along the lines of the AIDA model, it helped create Awareness/Attention, Interest, Desire, and Activation. However, more recent research, including behavioural science, econometrics, and media mix modelling, have shown that the picture is much more complex. One of the best studies of how advertising works is one carried out for the IPA by Les Binet and Peter Field, which produced the report “The Long and the Short of It”. Short Term and Long Term? One of the key findings in the work by Binet and Field is that short-term success is not a good or reliable indicator of long-term success. Rational measures, such as standout and attention are quite good at predicting […]

Why social media mining and monitoring have met with limited success in Market Research?

OK, let’s get one thing clear from the outset; I am not saying social media mining and monitoring (the collection and automated analysis of quantitative amounts of naturally occurring text from social media) has met with no success. But, I am saying that in market research the success has been limited. In this post I will highlight a couple of examples of success, but I will then illustrate why, IMHO, it has not had the scale of success in market research that many people had predicted, and finally share a few thoughts on where the quantitative use of social media mining and monitoring might go next. Some successes There have been some successes and a couple of examples are: Assessing campaign or message break through. Measuring social media can be a great way to see if anybody is talking about a campaign or not, and of checking whether they are talking about the salient elements. However, because of some of the measurement challenges (more on these below) the measurement often ends up producing a three level result, a) very few mentions, b) plenty of mentions, c) masses of mentions. In terms of content the measures tend to be X mentions […]

Using sampling error as a measure of reliability, not validity

Last week Jeffrey Henning gave a great #NewMR lecture on how to improve the representativeness of online surveys (click here to access the slides and recordings). During the lecture he touched lightly on the topic of calculating sampling error from non-probability samples, pointing out that it did not really do what it was supposed to. In this blog I want to highlight why I recommend using this statistic as a measure of reliability, but not validity. If we calculate the sampling error for a non-probability sample, for example from an online access panel, we are not representing the wider population. The population for this calculation is just those people who might have taken the survey. For example, just those members of the online access panel who met the screening criteria and who were willing (during the survey period) to take the study. The sampling error tells us how good our estimates of this population are (i.e. those members of the panel who met the criteria and who were willing to take a survey at that particular time). If we take a sample of 1000 people from an online access panel and we calculate that the confidence interval is +/-3% at […]

The differences between academic and commercial research

I am currently at an academic conference on mobile research in Cyprus, a WebDataNet event. I am a keynote speaker and my role is to share with the delegates the commercial market research picture. I really enjoy mixing with the academic world, and I am intrigued and fascinated by the differences between the academic and commercial worlds. This post looks at some of the key differences that I have noticed. Timelines In the academic world, timelines are usually longer than in market research. For example, an ethnographic project might be planned for 8 months, in the field for 4 months, and spend 12 months being analysed and written up. A commercial ‘ethnography’ might spend 4 weeks in design and set-up, the fieldwork might be wrapped up in 2 weeks, and the analysis and ‘write up’ conducted in 2 weeks. In many ways the differences in the timelines result from differences in the motivation for doing a research project. Commercial market research is often conducted to answer a specific business question, which means the research has to be conducted within the timeline required by the business question – which is typically rapid. Academic research is typically conducted to advance the body […]