We will be closing this survey, and starting the analysis on 30 January. The report and presentation will be released in February.
NewMR has partnered with Wizu to conduct research into the Opportunities and Threats facing Market Research and Insights in 2019. This project has two key aims:
1) To produce a report that seeks to explore the current perceptions of Opportunities and Threats facing market research and insights.
2) To explore the use of a chatbot for collecting this type of B2B information.
If you have not had a chance to take the chatbot survey yet, click here to take the survey (before reading further).
The Wizu Chatbot
For this project, we are using Wizu and are being assisted by Wizu’s Martin Powton. For more information about Wizu click here.
Report and Presentation
Once we have collected sufficient data, we will be producing and publishing a report and presenting our findings via a webinar – more information on both when we are closer to the date of publishing them.
Comparison Between a Chatbot and a Traditional Study
You can download a report of a study that Wizu have run comparing a traditional study with a chatbot survey by clicking here. Amongst the findings are a) the chatbot survey took longer to complete (3 mins instead of 2), b) the satisfaction with the chatbot survey experience was higher than that of the traditional survey.
At the moment we have 129 responses, but we are looking for something in excess of 300, so please do share the link to this page. We expect to close the survey about 23 January and we will present the results in February.
In terms of Optimism, with respect to market research and insights, just 19% have a Pessimistic outlook, with 47% selecting positive – and the remaining 35% being Neutral. Looking at the open-ended comments we see that:
- Negative predictions tend to relate to MR being stuck in the past, for example saying that perhaps MR is a dead end or is unwilling to change.
- The Neutrals tend to be unsure about the future (as opposed to predicting stability) – they can see pluses and minuses.
- The Optimists focused on the power of data, the role AI will play, and the need for data to be interpreted.
At the moment, the following five open-ends are good examples of what participants entered in terms of Threats:
- That no actionable insight is drawn – company data and market research data should be supportive of each other.
- What people actually do vs what they say they do.
- In house data.
- Automation Increasing access to data (behavioural data) and the increased interest in data science as the holy grail of data analysis and insight means that research teams are going to have to be more effective at selling why market research is important, the value of qualitative, and story-telling to great effect.
- Technology Invalid research design and execution.
At the moment, the following five open-ends are good examples of what participants entered in terms of Opportunities:
- Additional data sources.
- The opportunity to incorporate market research into more areas of the business as data becomes more connected.
- We do need to embrace all the other data available.
- If research suppliers and client teams can augment research capabilities with more evolved data analysis and mining skills, both with internal and external data, there’s an opportunity to create research that reflects a more holistic understanding of the business context and issue and may ultimately drive better strategic decision-making.
- Digital research.
In essence, almost all of the opportunities quoted by participants related to data and technology, whereas the threats included a much wider range of topics.
Reaction to the chatbot survey
The chatbot survey finished with open-ended comments about the topic and the survey. About 25% of these comments were evaluations of the chatbot – about 60% of these were positive and 40% were negative.
One of the interesting aspects of this experimentation is the use if intelligent probing. One of the hopes for AI-based interviews (including chatbot surveys) is that they will focus on the participant and ask relevant questions, rather than asking a pre-determined grid of everything that might be interesting. For some people, the probing worked and we got positive feedback. But when the ‘intelligence’ did not work it elicited negative comments. However, it is unlikely that we will get to a point where MR has 99% good probing withing using the current systems in real projects and using them to learn.
There were some interesting points of view about the idea of chatbots, including:
1) If a chatbot adds nothing other than making the survey experience more pleasant for the participant is it worth doing? Many people said yes, but quite a few people said it was irrelevant – “where has the bot added value?”.
2) Some people complained that the survey was too much like a survey and not a qualitative discussion. Other people complained that there were too many open-ended questions. Both groups of people will be able to design their surveys their way, this was one experiment, designed by us and Wizu.
3) One of the most interesting groups of criticisms was that it was obvious the chatbot was a computer and not a person. This raises an interesting methodological and ethical question about whether MR should ever use interview methods where the participant does not know they are talking to a machine?
There were quite a few questions about timing, costs, and the results, we will be addressing these in our reports.
More feedback soon.