Using a Word Cloud to analyse open-ended comments about AI and Automation in Market Research and Insights
Published by, Ray Poynter, 15 February, 2023
NewMR recently conducted a study into how market research and insight professionals view the impact of AI and automation over the next ten years. You can access our report on that study by clicking here.
In this post, I look at how I used Word Cloud Plus (our new, free, tool for creating insightful word clouds) to kick start my analysis of the open-ended responses to the question “What else would you like to tell us about AI or Automation?”
Cleaning the Text
Before starting to analyse the text, my first task was to tidy up the text, i.e. the open-ended responses to our question. This tidying up included things like fixing spelling mistakes and standardizing on US spellings (e.g. changing Colour to Color). Since the data was already in Microsoft Excel, I used the spellchecker that is built into Excel to do my tidying.
Regardless of which word cloud generator you are using, you will find that tidying up your source data will often improve the cloud you generate and the analysis you can do.
An Initial Word Cloud
My first step in the analysis was to generate a word cloud with all of the available text. For this project, the initial word cloud is shown below.
The next step was to remove words that were not part of the story, i.e. words that are not helping create the insights. For example, when I ask people about AI and automation they use the words AI and automation in their answers. Consequently, the terms AI and automation are not very useful in the word cloud. The way we remove terms like this is to make them into ‘Stop Words’. Stop words are words that do not appear in the cloud. Words such as ‘is’ and ‘and’ are set to be Stop Words by default.
Once I removed the words I had used in the question and did some tidying, I did my initial investigations. In the initial stage of my investigations, I looked at the words in larger fonts (the ones that occur the most often) and assigned colours to similar words and moved some of the words around to group similar topics – particularly those relating to people and humans. This revised cloud is shown below.
The next step was to use the themes in the word cloud to suggest groups of answers to read, to identify the messages in the text. In Word Cloud Plus, this means clicking on a word in the word cloud and then drilling down to the raw, open-ended comments that link to that word. Note, a word cloud does not remove the need to read the text. Word Cloud Plus gives you a better starting point, groups similar statements together, and reduces the time taken to form a view.
After reviewing the topics highlighted in the word cloud and reading the underlying comments, I identified three core messages and some negatives. The three core messages relate to The Human Element, Potential and Grunt Work, and Transforming the Industry.
The Human Element
One of the most common themes in the responses was the feeling that humans would still be necessary, for example, “but the human touch will be still necessary” and “it’s likely to stay most useful as an adjunct to human understanding and interpretation”. This view that the human element will still be necessary seems to be built on a combination of hope and an assumption that AI will remain (for the foreseeable future) unable to interpret humans in the way that trained/skilled researchers can.
Potential and Grunt Work
The potential of AI is centered on its ability to tackle repetitive tasks, such as the coding of open-ended responses. People see AI and automation as providing the opportunity to do more with less (with less time, less effort, fewer people, less money etc). Examples include “it will continue to help take away effort and grunt work which is a good thing for talented researchers!”, “ai has great potential to save time and money in some areas that are currently labor and time intensive” and “ai seems well suited to scalable measurement of ‘subjective’ stuff–like open ends”.
However, not everybody is convinced by the ability of AI to handle the coding of open-ended comments. For example, “I’m a skeptic! after using both commercial and in-house ai tools to analyze open ends”. I wonder if the sceptics are basing their reservations about tomorrow’s AI and automation on the basis of yesterday’s tools?
Transforming the Industry
There is widespread agreement that AI and automation will transform the research industry. However, there is less agreement about what the transformation will look like. Some see market research as simply part of the larger picture “If ai is predicted to disrupt virtually every industry how can research be an exception to this?”. Some highlight the risk of sameness and reduced roles for humans “homogenizing and also worrying in terms of the impact on jobs in our industry and others”. And the dystopian, “Could our industry eventually be robots talking to robots?”
Negatives
There are some specific negatives that people worry about. I’ve mentioned concerns about the quality of interpretation earlier, but there are also people talking about the impact on the quality of the data we are using, for example, “It may encourage the use of fraudulent bots in quantitative research!” and “How will researchers ever know if open ends were generated by computer or person.
The Big Picture?
Beyond a few sceptics who feel it is mostly hype, the consensus is that AI and automation are changing our industry. The positives tend to focus on removing the grunt work and freeing researchers to be more like consultants. More generally, AI and automation are felt to be likely to make research faster and cheaper – but few people thought it would make it better.
The people we spoke to were divided on how quickly they thought the change would happen, from almost immediately to more than ten years. The responses also showed a wide variation in how good they think the AI and automation tools will be. The sceptics (who felt it would be worse than today’s research) seem to outnumber the optimists (who mostly felt that AI research could be as good as today’s research and more scalable).
My view is that AI and automation will happen quicker than most people think, with the DIY platforms leading the way. My experience of research over the last 45 years tells me that if a solution is faster, cheaper but not quite as good, then it will do very well.
The Future
I plan to come back to this topic in 2024, and I expect to see significant changes, and hopefully some progress.
Do you want to try Word Cloud Plus?
Word cloud is free to use. If you want to try it out with a data set of your own, go to Word Cloud Plus, create a free account and get started.
If you want to borrow a set of data, you can access a set of verbatims from an old NewMR study looking at what makes a presenter a good presenter, click here to access the file.