3 counterintuitive but practical tips for chatbot survey design

Guest post by Jennifer Reid, the Senior Methodologist at Rival Technologies, 12 March 2019

---

In his annual predictions for 2019, Ray Poynter of NewMR singles out chatbots as an interesting trend to watch. “I think 2019 is going to be a major breakthrough for chatbots,” he writes, adding that companies should consider doing pilot projects in the first half of this year.

Ray is definitely on to something. One study by Oracle shows that 80% of business leaders want to integrate chatbots in their processes. Consumers seem keen on chatbots as well—in another study, 80% of consumers report having positive experiences with chatbots.

At Rival Technologies, a big part of my role as Senior Methodologist is to ensure that the solutions we are bringing to the market have a positive impact both for insight professionals and research participants. Chatbot surveys have become a core product of our platform, so I’ve had the opportunity to really dive in and get my hands dirty with this new way of engaging with consumers.

One thing we learned very quickly is that many best practices in traditional survey design don’t necessarily apply to chatbot surveys (or “chats,” as we like to call them at Rival). If you’re simply copying your online surveys in your chats, you’ll lose people’s attention very quickly.

Creating effective chats isn’t hard—but it requires taking a step back and re-examining common survey design practices. As a starting point, here are three tips you should consider if you’re doing any chatbot pilots this year.

Tip #1: Avoid starting with demographic info.

A commonly accepted best practice in market research is to ask for demographic info at the beginning of the survey. The thinking is that this ensures you’re talking to the right people and you can disqualify those who are outside of your target audience.

But this long-standing practice doesn’t work in chats. In fact, if your first question is about a person’s age, marital status, gender or income, you’ll see a very high drop off. Some groups (young consumers, for example, who aren’t as familiar with the experience of traditional surveys) may find it suspicious that you are asking for personal information to begin with.

What we find more effective is to lead with your most engaging questions—something that most people would be happy to answer. Think about how you’d naturally open a conversation with a friend or an acquaintance, and then emulate that approach in your chat.

If you really need to start with demographics because of quotas and other business requirements, I’d suggest explaining why you need this information. Don’t assume that people who take chats over Messenger or SMS know about your intentions.

Tip #2: Don’t pretend to be a human.

When designed right, chatbots can make the survey-taking experience feel more human and less clinical. Through emojis, GIFs and videos, chats can feel more like a conversation with a friend rather than an interrogation with a face-less brand.

That said, chats should not deceive respondents into believing that they are talking to a real person. People should clearly know they are interacting with a chatbot rather than a real human. Some jurisdictions such as California are starting to put laws in place requiring chatbots to disclose that they are not human. I recommend telling respondents upfront that they are interacting with a chatbot—perhaps in the invitation or the first few messages in your chat.

Tip #3: Shorten—even more than usual

Long surveys are still, unfortunately, a reality in our industry. It’s a big problem because, as a GRBN study recently found out, long surveys are harming the reputation of the market research industry.

From our experience, 70% of chats are completed via mobile devices. Chat respondents, by virtue of being mobile, can be anywhere. They could be completing the chat while at a hockey game, on the train on their way home or during a break at work. Often, mobile respondents don’t have a lot of time, so a 20-minute survey is out of the question.

Also, consider the channel by which respondents receive chat invitations: Facebook Messenger or SMS. When people use these platforms to talk with friends and family, they do so in small chunks of time. People tend to chat for a few minutes, leave the conversation, and then go back in when they’re ready. This chat-leave-chat type of interaction requires very short activities.

In the mobile-first era, long surveys are dead. Keep your chats under 10 minutes.

Bottom line: Don’t just copy your online surveys!

It’s counterintuitive but chat surveys are most effective when they don’t feel like a survey and feel more like a conversation. The good news is that taking the time to design respondent-first chat surveys is well worth the effort. From our experience, typical response rates for chats are 50-70%. Subscription rates (that is, the ratio of people who agree to answer future chats) are up to 80%.

As I discussed in a recent research-on-research, a big reason for these high numbers is the respondent experience. People love the conversational aspect of chats. For example, 88% of participants told us they found the chat experience either “much more” or “somewhat more” enjoyable than other surveys they’ve taken in the past. 

At Rival, we are committed to developing this emerging technology and helping research and marketing leaders get the most out of it. We’ll be sharing more findings and best practices in the future. In the meantime, feel to check out our free ebook, Create Effective Chat Surveys, for more tips.

---

About the author: Jennifer Reid is the Senior Methodologist at Rival Technologies, a Vancouver-based software company developing voice, chat and voice research solutions for the mobile-first generation. In her role at Rival, she leads the charge in the development of chat surveys and other conversational research technologies. Jennifer also serves on the Board of Reid Campbell Group and of Angus Reid Enterprises.