Eleven Things Research Pros Should Know about Sentiment Analysis

Guest post by Seth Grimes. Seth Grimes is an analytics strategy consultant with Washington DC based Alta Plana Corporation. He is founding chair of the Text Analytics Summit (2005-13), the Sentiment Analysis Symposium (next event July 15-16, 2015 in New York), and the LT-Accelerate conference (November 25-26, 2015 in Brussels).


Sentiment analysis has been portrayed, variously, as a linchpin of the New MR and as snake oil that produces pseudo-insights that are little better than divination. Who’s right?

Me, I’m with the first camp. Automated sentiment analysis, powered by text analytics, infuses explanatory power into stale Likert-reliant methodologies and allows researchers to stay on top of fast emerging trends, and to tap the unprompted voice of the customer, via social listening.

EmoticonsI suspect that most in the second, nay-sayer camp have distorted ideas of sentiment analysis capabilities and limitations. These ideas have perhaps been engendered by extravagant and unsupported claims made by less-than-capable solution providers. Whatever their source, I’ve take it on myself to debunk them, to give a truer sense of the technology.

We aim to encourage appropriate use of sentiment technologies and to discourage their misuse. Call the effort market education. I do a lot of it, via conferences such as my up-coming Sentiment Analysis Symposium, taking play July 15-16 in New York, and via articles such as this one, which offers —

Eleven things research pros should know about sentiment analysis:

1) Sentiment analysis via term look-up in a lexicon is an easy but crude method. Meaning varies according to word sense, context, and what’s being discusses. Look for methods that apply linguistic and statistical methods to the analysis task.

2) Document-level sentiment analysis is largely passé. Aim for sentiment resolution at the entity, concept, or topic level. (Examples: An Apple iPhone 6 is an entity; the iPhone line is a conceptual category; smart phones are a topic.)

3) The common-language definition of ‘sentiment’ includes attitude, opinion, feelings, and emotion. Capable sentiment analysis will allow you to go beyond positive/negative scoring to allow rating according to emotion — happy, surprised, afraid, disgusted, angry, and sad — and mood and not just valence.

4) Expanding on that broad view: Sentiment analysis is part of the world of affective computing, “computing that relates to, arises from, or deliberately influences emotion or other affective phenomena,” quoting the MIT Media Lab’s Affective Computing group. Contrast with cognitive and sensory computing: All linked, but with distinctions in the technologies and methods applied.

5) Not all sentiment is created equal. You should strive to understand both valence and intensity, and also significance, how sentiment translates into actions.

6) Whether you apply language engineering, statistical methods, or machine learning to the task, properly trained domain-adapted models will outperform generic classification.

7) You need to beware of accuracy claims. There’s no standard measuring stick, and some solution providers even cook the measurement process. The accepted approach is to measure accuracy against a “gold standard” of human annotated/classified material. That means setting humans and machines on the same tasks and seeing the degree of agreement. But if you have your software take a shot at the task, and then have a human decide whether it was right of not, that’s not legit. And no standard measuring stick: Some software does only doc level analysis while other software analyzes at the sentence or phrase level, and yet other software resolves sentiment to particular entities and concepts. Maybe 70% at the entity level is better than 97% at the doc level?

8) Text is the most common sentiment data source, but it’s not the only one. Apply facial coding to video, and speech analysis to audio streams, in order to detect emotional reaction: These are advanced methods for assessment of affective states. The next frontier: Neuroscience and wearables and other means of studying physiological states.

9) Language is among the most vibrant and fast-evolving tools humans use. Personal and social computing have given us unprecedented expressive power and ability to amplify our voices, via old-new methods such as emoji. More than just nuanced amplifiers — 😀 vs. 😈— emoji have taken on syntax and semantics of their own, and of course social media is awash in imagery. Sentiment analysis is keeping pace with the emergence of new forms of expression. (A plug: I’m particularly excited about a pair of Sentiment Analysis Symposium presentations in this area. Francesco D’Orazio, of UK agency FACE and Pulsar Social, will speak on Analyzing Images in Social Media, and Instagram engineer Thomas Dimson will be speaking on the semantics of emoji, on “Emojineering @ Instagram.” Other symposium presenters cover others topics I’ve mentioned in this article.)

10) You can gain analytical lift, and predictive power, by linking sentiment and behavior models, and by segmenting according to demographic and cultural categories. There’s lots of data out there. Use it. Here’s why —

11) Advanced concepts such as motivation, influence, advocacy, and activation are built on a foundation of sentiment and behavioral modeling and network analysis. If the goal of research, in the insights industry, is consumer and market understanding, the goal of understanding is to create the conditions for action. You need to work toward these concepts.

Alright, there’s my take. Consider it when you design your next survey — don’t shy away from free-response verbatims — and as you wonder how to bring social-media mining into your studies. Think about the variety of affective-computing methods available to you and which might help you, in conjunction with behavior analyses and more advanced segmentation, generate insights that your clients can act on. Market researchers and insight pros, relook sentiment analysis in order to add New to your MR.

4 thoughts on “Eleven Things Research Pros Should Know about Sentiment Analysis

  1. Given the comments here one gets a sense that sentiment analysis is still a work in progress. At present it claims to be fairly accurate at linking comments to some very basic marketing indicators such as the Ekman seven basic emotions and at its most simplest positive/negative and some measures of valence. Not much there I would say. To what extent can analytics be prescribed to sentiments as links to behavioural data? I think that’s where everyone is
    waiting for the validation. I haven’t seen much of that as yet. To postulate potential links to motivation and activation are pretty strong claims, but I guess some of us live in hope. I hope it comes to something!

  2. Chris, you write, “I think that’s where everyone is waiting for the validation. I haven’t seen much of that as yet.”

    This reminds me of the exchange in the Tempest. Miranda declaims, on seeing people, other than her father Prospero, for the first time —

    “O wonder! / How many godly creatures are there here! / How beauteous mankind is! O brave new world, / That has such people in’t.”

    Prospero’s reply –“‘Tis new to thee.”

  3. Haha. Made me think! But seriously Seth there isn’t anything published other than anecdotal stories about the power of this stuff and a wide range of companies using it, or should we still say testing it? And to be honest I have done a lot of reading on this so hardly a “Miranda”. Even Ekman started to explore more than 7 basic emotions and a lot of criticism from the neuroscience area focuses on the fact that measures are all frontal not temporal and therefore miss out on finer detail about emotions. The problem with this stuff is it is so available and pictorial that I wonder if the analysis becomes more post rationalized than real and is just another form of confirmation bias. To quote Shakespeare again, and from Macbeth, one hopes that this is indeed not “a tale told by an idiot, full of sound and fury, signifying nothing”

  4. Chris, you suspect a null pointer, as programmer might say. Or since this is an “insights” board, should “signifying nothing” branch us off into semiotics?

Comments are closed.