Why social media mining and monitoring have met with limited success in Market Research?

Shibyu At Night

OK, let’s get one thing clear from the outset; I am not saying social media mining and monitoring (the collection and automated analysis of quantitative amounts of naturally occurring text from social media) has met with no success. But, I am saying that in market research the success has been limited.

In this post I will highlight a couple of examples of success, but I will then illustrate why, IMHO, it has not had the scale of success in market research that many people had predicted, and finally share a few thoughts on where the quantitative use of social media mining and monitoring might go next.

Some successes
There have been some successes and a couple of examples are:

Assessing campaign or message break through. Measuring social media can be a great way to see if anybody is talking about a campaign or not, and of checking whether they are talking about the salient elements. However, because of some of the measurement challenges (more on these below) the measurement often ends up producing a three level result, a) very few mentions, b) plenty of mentions, c) masses of mentions. In terms of content the measures tend to be X mentions on target, or Y% of the relevant mentions were on target – which in most cases are informative, but do not produce a set of measures that have any absolute utility and usually can be tightly aligned with ROI.

An example of this use came with the launch of the iPhone 4 in 2010. Listening to SM made it clear that people had detected that the phone did not work well for some people when held in their left hand, that Apple’s message (which came across as) ‘you should be right handed’ was not going down well, and that something needed to be done. The listening could not put a figure on how many users were unhappy, nor even if users were less or more angry than non-users, but it did make it clear that something had to be done.

Identifying language, ideas, topics. By adding humans to the interpretation, many organisations have been able to identify new product ideas (the Nivea story of how it used social media listening to help create Nivea Invisible for Black and White is a great example). Other researchers, such as Annie Pettit, have shown how they have combined social media research with conventional research, to help answer problems.

Outside of market research. Other users of social media listening, such as PR and reaction marketers appear to have had great results with social media, including social media listening. One of the key reasons for that is that their focus/mission is different. PR, marketing, and sales do not need to map or understand the space, they need to find opportunities. They do not need to find all the opportunities, they do not even need to find the best opportunities, they just need to find a good supply of good opportunities. This is why the use of social media appears to be growing outside of market research, but also why its use appears to be in relative decline inside market research.

The limitations of social media monitoring and listening
The strength of social media monitoring and listening is that it can answer questions you had not asked, perhaps had not even thought of. Its weakness is that it can’t answer most of the questions that market researchers’ clients ask.

The key problems are:

  • Most people do not comment in social media, most of the comments in social media are not about our clients’ brands and services, and the comments do not typically cover the whole range of experiences (they tend to focus on the good and the bad). This leaves great holes in the information gathered.
  • It is very hard to attribute the comments to specific groups, for example to countries, regions, to users versus non-users – not to mention little things like age and gender.
  • The dynamic nature of social media means that it is very hard to compare two campaigns or activities, for example this year versus last year. The number of people using social media is changing, how they are using it is changing, and the phenomenal growth in the use of social media by marketers, PR, sales, etc is changing the balance of conversations. Without consistency, the accuracy of social media measurements is limited.
  • Most automated sentiment analysis is considered by insight clients and market researchers to either be poor or useless. This means good social media usage requires people, which tends to make it more expensive and slower, often prohibitively expensive and often too slow.
  • Social media deals with the world as it is, brands can’t use it to test ads, to test new products and services, or almost any future plan.

The future?

Social media monitoring and listening is not going to go away. Every brand should be listening to what its customers and in many cases the wider public are saying about its brands, services, and overall image. This is in addition to any conventional market research it needs to do; this aspect of social media is not a replacement for anything, it is a necessary extra.

Social media has spawned a range of new research techniques that are changing MR, such as insight communities, smartphone ethnography, social media bots, and netnography. One area of current growth is the creation of 360 degree views by linking panel and/or community members to their transactional data, passive data (e.g. from their PC and mobile device), and social media data. Combined with the ability of communities and panels to ask questions (qual and quant) this may create something much more useful that just observational data.

I expect more innovations in the future. In particular I expect to see more conversations in social media initiated by market researchers, probably utilising bots. For example, programming a bot to look out for people using words that indicate they have just bought a new smartphone and asking them to describe how they bought it, what else they considered etc – either in SM or via asking them to continue the chat privately.
There are a growing number of rumours that some of the major clients are about to adopt a hybrid approach, combining nano-surveys, social media listening, integrated data, and predictive analytics, and this could be really interesting, especial in the area of tracking (e.g. brand, advertising, and customer satisfaction/experience).

I also expect two BIG technical changes that will really set the cat amongst the pigeons. I expect somebody to do a Google and introduce a really powerful, free or almost free alternative to the social media mining and monitoring platforms, and I expect one or more companies to come up with sentiment analysis solutions that are really useful. I think a really useful platform will include the ability to analyse images and videos, to follow links (many interesting tweets and shares are about the content of the link), to build a PeekYou type of database of people (to help attribute the comments), and will have much better text analytics approach.


4 thoughts on “Why social media mining and monitoring have met with limited success in Market Research?

  1. I of course disagree with some of your comments as they apply equally to other methodologies (e.g., not everyone joins panels, social media penetration is evolving as the survey method did before) but your overall point is well taken.

    I believe that there are two major issues
    1) People are surprised by the pricing. Though internet data is ‘free,’ creating valid and reliable data collection, cleaning, scoring, coding, and presentation processes requires people power. Which brings the pricing in line with surveys. Not free. Not cheap.
    2) Social media data looks different and acts different, and surveys are already doing a pretty good job. Why change what we’re doing when it’s easier to keep on doing what we’ve been doing. We haven’t got time to wrao our heads around something new.

    Discuss 🙂

  2. Hi Ray, thanks for posting this, it is an important discussion. I agree with Annie that Good social media analytics, like good research, requires brains as well as technology which are hard to replicate with an algorithm. Particularly important is the last mile where the data needs to be connected to the business need. Computers are never very good at that.

    I think there are some other issues to consider as well:

    1. The quality of the results are highly dependent on the quality of data and the quality research design. For example many of the richest discussions in social data exist in forums (like http://www.justmommies.com/forums/) but most of the social data tools do a poor job of collecting and cleaning this data.

    2. Different social sources require different analytical approaches. For example Twitter data is like a Mall intercept, short answers, limited depth of insight, forum data is more like analyzing a friendship focus group, with deep discussions and answers. Without the right design approach the output will be flawed.

    3. In many categories brand mentions are low. For example in food only 5% of any conversations mention any brand. The fact that few people are talking about your brand is the insight of the research. The role of the research analysts is to help the client discover what they ARE talking about. To look at what discussions have passion and to see what area of passion a brand can legitimately “Lasso” with their DNA. This is the way to get people to talk about your brand.

    Campaign monitoring is just the after measurement. Category analytics is the key to better strategy before the campaign launches. However again, most of the social tools are very week when it comes to category vs brand analytics as it requires a lot more sophistication and computing power.

    4. Looking back in time to examine the longitudinal changes in customer behavior is another data issue. if you have been collecting and storing the data for long enough then it is easy to look back in time and compare say the launch of the Xbox this time to the last model launch and benchmark accordingly. Again a challenge is that many tools do not have data saved for long enough to make this easy.

    In conclusion I think that just as it is hard to generalize that all research is useful or all research is not it is equally hard to generalize that all social research is good or bad. Great social research, IMHO, has been embraced by great client researchers at the best Fortune 500 companies. Sadly there is a lot less great social market research and social research tools right now than the other kind.

Comments are closed.