Eleven Things Research Pros Should Know about Sentiment Analysis

Guest post by Seth Grimes. Seth Grimes is an analytics strategy consultant with Washington DC based Alta Plana Corporation. He is founding chair of the Text Analytics Summit (2005-13), the Sentiment Analysis Symposium (next event July 15-16, 2015 in New York), and the LT-Accelerate conference (November 25-26, 2015 in Brussels).

---

Sentiment analysis has been portrayed, variously, as a linchpin of the New MR and as snake oil that produces pseudo-insights that are little better than divination. Who’s right?

Me, I’m with the first camp. Automated sentiment analysis, powered by text analytics, infuses explanatory power into stale Likert-reliant methodologies and allows researchers to stay on top of fast emerging trends, and to tap the unprompted voice of the customer, via social listening.

I suspect that most in the second, nay-sayer camp have distorted ideas of sentiment analysis capabilities and limitations. These ideas have perhaps been engendered by extravagant and unsupported claims made by less-than-capable solution providers. Whatever their source, I’ve take it on myself to debunk them, to give a truer sense of the technology.

We aim to encourage appropriate use of sentiment technologies and to discourage their misuse. Call the effort market education. I do a lot of it, via conferences such as my up-coming Sentiment Analysis Symposium, taking play July 15-16 in New York, and via articles such as this one, which offers —

Eleven things research pros should know about sentiment analysis:

1) Sentiment analysis via term look-up in a lexicon is an easy but crude method. Meaning varies according to word sense, context, and what’s being discusses. Look for methods that apply linguistic and statistical methods to the analysis task.

2) Document-level sentiment analysis is largely passé. Aim for sentiment resolution at the entity, concept, or topic level. (Examples: An Apple iPhone 6 is an entity; the iPhone line is a conceptual category; smart phones are a topic.)

3) The common-language definition of ‘sentiment’ includes attitude, opinion, feelings, and emotion. Capable sentiment analysis will allow you to go beyond positive/negative scoring to allow rating according to emotion — happy, surprised, afraid, disgusted, angry, and sad — and mood and not just valence.

4) Expanding on that broad view: Sentiment analysis is part of the world of affective computing, “computing that relates to, arises from, or deliberately influences emotion or other affective phenomena,” quoting the MIT Media Lab’s Affective Computing group. Contrast with cognitive and sensory computing: All linked, but with distinctions in the technologies and methods applied.

5) Not all sentiment is created equal. You should strive to understand both valence and intensity, and also significance, how sentiment translates into actions.

6) Whether you apply language engineering, statistical methods, or machine learning to the task, properly trained domain-adapted models will outperform generic classification.

7) You need to beware of accuracy claims. There’s no standard measuring stick, and some solution providers even cook the measurement process. The accepted approach is to measure accuracy against a “gold standard” of human annotated/classified material. That means setting humans and machines on the same tasks and seeing the degree of agreement. But if you have your software take a shot at the task, and then have a human decide whether it was right of not, that’s not legit. And no standard measuring stick: Some software does only doc level analysis while other software analyzes at the sentence or phrase level, and yet other software resolves sentiment to particular entities and concepts. Maybe 70% at the entity level is better than 97% at the doc level?

8) Text is the most common sentiment data source, but it’s not the only one. Apply facial coding to video, and speech analysis to audio streams, in order to detect emotional reaction: These are advanced methods for assessment of affective states. The next frontier: Neuroscience and wearables and other means of studying physiological states.

9) Language is among the most vibrant and fast-evolving tools humans use. Personal and social computing have given us unprecedented expressive power and ability to amplify our voices, via old-new methods such as emoji. More than just nuanced amplifiers — 😀 vs. 😈— emoji have taken on syntax and semantics of their own, and of course social media is awash in imagery. Sentiment analysis is keeping pace with the emergence of new forms of expression. (A plug: I’m particularly excited about a pair of Sentiment Analysis Symposium presentations in this area. Francesco D’Orazio, of UK agency FACE and Pulsar Social, will speak on Analyzing Images in Social Media, and Instagram engineer Thomas Dimson will be speaking on the semantics of emoji, on “Emojineering @ Instagram.” Other symposium presenters cover others topics I’ve mentioned in this article.)

10) You can gain analytical lift, and predictive power, by linking sentiment and behavior models, and by segmenting according to demographic and cultural categories. There’s lots of data out there. Use it. Here’s why —

11) Advanced concepts such as motivation, influence, advocacy, and activation are built on a foundation of sentiment and behavioral modeling and network analysis. If the goal of research, in the insights industry, is consumer and market understanding, the goal of understanding is to create the conditions for action. You need to work toward these concepts.

Alright, there’s my take. Consider it when you design your next survey — don’t shy away from free-response verbatims — and as you wonder how to bring social-media mining into your studies. Think about the variety of affective-computing methods available to you and which might help you, in conjunction with behavior analyses and more advanced segmentation, generate insights that your clients can act on. Market researchers and insight pros, relook sentiment analysis in order to add New to your MR.