3 X 3 – 9 Ways to Win with Qualitative Online Communities in 2021

Click here to access the slides

Transcript of recording with Holly M O’Neill – generated automatically by HappyScribe which means it will be about 80% accurate – if you spot confusing errors, please email ray@new-mr.com. The timestamps are included to help you jump directly to a point of interest.

[00:00:08.340] – Holly M O’Neill

Hello, everyone, I’m really excited to be here today to talk about ways to win with qualitative online communities. It is my favorite qualitative methodology and I’ve been doing asynchronous online community since two thousand and seven. And I am so excited by the amount of technology transformation that’s taken place in the last decade or so, the new research tools and techniques and and moderator dashboard functionality and segmentation really makes us a fabulous type of methodology. Fun fact about me. I tried to win the hundred thousand dollar pyramid on the very last day.

[00:00:47.610] – Holly M O’Neill

They were on the air before they took a 10 year hiatus. So on today’s agenda, I really want to talk about tips and tricks for your toolkit. I’m a big believer in having a large qualitative toolkit full of exercises, ideas and approaches. And with these you can pick and choose whatever relevant to the project that can create a really highly customized research plan and discussion guide for your online communities. Whether they’re short term commodities or long term communities, the exercises that and the techniques I’m going to discuss today work in long term communities for all of them, and most of them apply to short term communities as well.

[00:01:31.740] – Holly M O’Neill

So let’s talk first about approaches to research plans, and the first one I want to suggest to you is think about could your testing be better if you set up random cells? I want to suggest the idea of injecting random cell testing into your online communities. The healthy part is quite an easy when you’re onboarding your members, you can assign them random cells. And if you have a longer term community, think about whether you need to evenly populate the cells based on participation or tenure of the members so that you can have even testing across.

[00:02:11.140] – Holly M O’Neill

This idea works for various testing needs. For instance, in the first example, if you have test concepts that are quite similar, such as a 30 second commercial and a 15 second commercial of the same treatment, if you showed both to the same respondent, you might have some bias. So in this case, the idea is to create two cells, show each respondent only one of the versions, and then analyze across cells. What you’ll see is intensity of findings.

[00:02:42.840] – Holly M O’Neill

This works great for maybe a full website landing page or just the part that shows above the fold. The intensity of the findings will come across in the qualitative discussions. Another example that moderators see often is the client has more versions than the time allowed for testing could take place. So in this scenario, we need to actually rotate the concept. It can be for any number that you have. Just this example. If you have six treatments and three cells, you could show each different version twice, two to two of the three cells.

[00:03:19.020] – Holly M O’Neill

Not everybody has to see everything and each one would get shown four times. The trick is just to make sure you divide evenly so that everything gets equal exposure. And that really helps with the respondent fatigue. That can help. That can happen with any kind of research testing. Next, I want to talk about what about using some of the tools inside the platforms a little bit differently. What if we want to test reverse segmentation in this example, the client in a category where the retailers sort of restrict or have constructs for how the segmentation is to take place?

[00:04:01.670] – Holly M O’Neill

For instance, let’s just say it’s serial, although this is a sanitized example, some retailers already have the shelf sets in the planogram. There’s the healthy section, the fruity section, the gluten free section, the value priced section, et cetera. So in this example, my client had three different segmentation hypotheses that were bounded by some of the limitations at retail. So they came up with three different schemas. And our goal was to test which of these schemas had the most comprehension and the most suitability with their customers.

[00:04:38.510] – Holly M O’Neill

We actually asked them to move each product into the category where they felt it fit best and then we compared well, they placed the items versus the client generated hypotheses. We asked a lot of open around this. Were there any specific cards that you could not sort easily overall with sorting easy or difficult? And which of the three schemas made the most sense for you for this category? And we had a lot of illuminating ideas about how the client hypotheses in some cases were working and in some cases not working.

[00:05:16.650] – Holly M O’Neill

Another idea is to send members outside of the community for testing first and then have them come back to the community for the quality of discussions, we could send them to the client website or the app, have them log in, have them not log into their account. If you need a personalized experience for your testing, you will need them to log in. Maybe you have something baited, a test or a clickable prototype. Any of these things are right for testing within a community because the members are already on the Internet.

[00:05:48.240] – Holly M O’Neill

One really key tip is that the how to for this needs to be very, very specific in many cases. And if it’s a web activity that might involve several steps, you might even need to give different instructions depending on if they’re using a mobile app or a desktop site, you probably also want to think about do you need to offer the participants an opt out? And should that be easy because some members may not want to actually log into their personalized account to do this testing.

[00:06:18.440] – Holly M O’Neill

And if you do give an opt out and it’s a long term community with a points and badge system, you might want to think about how that opt out would affect their points for the month or the time frame and the program as to how you award those incentives. If you are going to use a link to go to a different site, make sure you can spawn it in a new window. So it’s easy for members to return to the community to that exact page, to come back into the discussions no matter where you send them, and also tell them up front that this one might take longer than normal and set some time expectations for them as well on this one.

[00:06:56.020] – Holly M O’Neill

Next, I want to get into a couple of exercises and perspectives that kind of work in various categories and for various needs, the first one is about co creation. Co creation is a real hot buzzword in research and innovation these days. And a a line of questioning that I really love Inside Communities is co creating with the members as if they are the celebrity. Give them permission to think bigger, let them think in the way the unique personality of a celebrity.

[00:07:27.560] – Holly M O’Neill

You might think they allow them to generate more ideas than their own usage or their own mindset might do. This works great in both short and long term communities, and I really like to give the participants their choice of celebrities. That way they have a little more skin in the game. It’s more relevant to them and they can get more into the exercise and fully engage with it. If you were Oprah or Dr. House or Sarah Jessica Parker, and what ways might you create whatever this new product is that would help your consumers do whatever it is you’re trying to solve and make sure your probes cover what it is, how it does it?

[00:08:07.000] – Holly M O’Neill

One, would you use it? And I find that consumers have a lot of fun with this. And I’m always shocked at how creative even everyday consumers are with this for the most part. And you could use this as a one off exercise or you could build it into a series of creation exercise where you might run two or three or four in a series to create a bunch of idea nuggets from your consumers. And I also really think it’s great to play them back, give these idea nuggets back to the to the same members and ask them to evaluate them and tell them someone in the community came up with the idea maybe it was you and actually test them back.

[00:08:46.670] – Holly M O’Neill

That’s another great engagement step that the co creation ideas like this can have in the community for engagement as well. Another idea that works really well in our online community is that we have so many multimedia opportunities and multimedia technology is used for visual metaphors, really helping consumers get into the right brains. That’s where the brand relationships are held. It seems lately I’ve been getting a lot of client calls for as they’re testing their new product ideas, regardless of whether it’s a baby idea very early on or it’s a almost a finished concept.

[00:09:24.120] – Holly M O’Neill

Is this does that have good brand? Do they have permission to go into that new category or that new segment? And by giving the members a choice of something that has a size connotation and asking them, can the brand get from where they are today to where they want to go with this new product idea, how far of a gap do they have to cross? Is it close then like the Atlantic City boardwalk, it’s a little bit bigger than a sidewalk or is a giant like the Grand Canyon.

[00:09:57.360] – Holly M O’Neill

And if you get a lot of Grand Canyon comments back, you know, your brand is going to have poor fit or a lack of permission for this type of new product. And you can really ferret that out then in the open in the discussion that goes with it. Be creative, choose any three objects that are substantially different in size, but also are highly visual. And a couple of extra tips to help with your analysis. You could concurrently run this as a poll or even a markup tool heat map exercise, because that will allow you to get through your analysis quicker.

[00:10:32.640] – Holly M O’Neill

And it will also give the opportunity for the respondents to put a stake in the ground so they can better articulate their reasoning. Back to you as to why they’re choosing Atlantic City, the Mississippi River or the giant Grand Canyon. Another exercise I love is using Venn diagrams and a lot of the online community platforms have wonderful tools and sometimes we think of using them just sort of in almost what they’re sort of geared toward or loosely designed for. Think of markup testing for maqam web pages and print ads or other kinds of communications and maqam.

[00:11:15.170] – Holly M O’Neill

But what if we use them with a Venn diagram about attributes and especially in categories that sort of have binary uses? For instance, we could ask, do you use your toaster oven primarily for toasting bread items or for heating frozen foods? Is that protein bar healthy or delicious? I have asked that one multiple times in multiple categories and that always brings back where the bean diagram a lot of healthy discussion, no pun intended. How about in terms of users is forever.

[00:11:49.190] – Holly M O’Neill

Twenty one for teenage girls hoping to become twenty one soon, or is it for older women hoping to look and feel twenty one again? I actually asked my niece this question one time when she was in college and she said to me that forever twenty one was for women her age, but it was OK if I wanted to shop there too, so that I could look cute. Anyway, this also works great for three way Venn diagram if you have a more complicated topic at hand.

[00:12:17.180] – Holly M O’Neill

But keep in mind that seven total sales and four overlapping sections, do you need to allow more time and make sure that it’s relevant to the consumers or the audience you have in testing? And also to let me throw off the idea of perceptual maps and grids work in the same way as well, using the heat mapping technology in a different way other than Markham testing. The last section I wanted to get into talking about boosting participation in this has more of a focus in the long term communities.

[00:12:51.920] – Holly M O’Neill

The first idea here is think about not only what can you get from the consumers, which is what we’re thinking about research wise, but what value can the brand provide to its members? Think about them more holistically. They’re just not research respondents, but they’re community members and they have holistic lives. And how can you share information with them or support the exchange of information beyond the research activities? In categories that are very information rich or where consumers often seek information such as babies, pets, cooking and entertaining, these categories are right for having.

[00:13:38.680] – Holly M O’Neill

Curated and repurposed content that the client already has on file sharing and the community in a fun and exciting way. Tips and hacks and recipes and glossaries and things of that nature. And also to the idea of open forums where the members just talk among themselves. These conversations can be unmoderated and for engagement purposes only. And I found that members find a lot of value in certain categories of conversing with like minded consumers. You know, you probably don’t have an immediate need or maybe not even the budget to analyze any of these open member only forums.

[00:14:18.340] – Holly M O’Neill

But I would suggest keeping them on file because they do come in handy for writing a discussion guide later on or maybe to help satisfy the emergency client request. And lastly, I would say think about getting the members involved in picking the topics, let them feel more included, let them feel more part of the community. I ran something in December where we had 12 topics for the open forums that were going to be run in twenty twenty one, and we let them pick the order that they wanted to see them, which were more popular for them, which were more important to them, and themed it around a certain rank with the twelve days of Christmas, as you can see, with a partridge in a pear tree card there.

[00:15:02.850] – Holly M O’Neill

You know, also one last tip on this page is that you have to be careful whatever you’re running an education or engage with it. It doesn’t conflict with whatever research topics you have running at the same time. Another way I like to engage my members is by having you be the moderator activity or an ask the moderator activity. I’ve been running the same online community for over ten years, so I have a unique rapport and bond with my members. And I recently fielded a to the moderator activity and it was a huge success.

[00:15:43.170] – Holly M O’Neill

The members got very engaged in asking me questions like I told them they could ask me whatever they wanted to ask me. And the question I got most was how and why did you become a moderator? I’ve never heard of this kind of job before I joined the community. What are your qualifications experience? What other kinds of projects do you work in? You they ask to be some community. InnerWorkings questions How long would it take to read all the posts?

[00:16:09.630] – Holly M O’Neill

How long does it take to write the reports? How are the topics chosen and how are the questions develop? They kind of wanted to know the nitty gritty of the day to day that the moderator does. They also asked me questions about the how I use the category, what kinds of soup do I like to make? And of course, they ask me how I and my family were doing the recovered and what we most look forward to when it was over.

[00:16:37.480] – Holly M O’Neill

And another idea I’ve got a lot of traction with lately is this idea of Friday pop ups. Now, this is really just for long term communities, but when you know how your participation varies by days of the week, you can boost participation by testing different launch days. A couple of ideas is a cross talking promotion of this could be. As a way to get them to understand how and why they should be posting on the comments of other participants, choose a topic that would make it easy for them to converse back and forth on other posts and also something that would be valuable to your analysis.

[00:17:16.950] – Holly M O’Neill

Of course, you could run it, say, Friday to Sunday. You could offer double points and make sure that you set a maximum points in your system. So there’s no gaming of the system depending on how you have your incentives system set up. Another idea is if you have a topic coming in early in the week and the whole new tangent has come up, something intriguing, something that wasn’t considered the objective stage, but you really want to go back and explore that in depth, more than just probing on comments could do or add a couple additional questions on a Friday called a hot off the press and really get a deep dove and whatever that new and exciting tangent is, or that you can have those fresh learnings and in-depth learnings on that exciting tangent that came up.

[00:18:04.310] – Holly M O’Neill

Another idea is to release your topics early in one of my communities, we almost always launch new topics on Monday, but what if we launch it the Friday before? And what we saw that even though we typically get the highest participation on Monday, Tuesday, Wednesday, we were able to push more than 60 percent of posts into the weekend. And that meant we had our results faster, which, of course, the client was super excited about to have that expedited learning, and it gave something new and exciting for the members, something that’s a little bit different than what they’re normally used to because it was mixing it up for excitement.

[00:18:46.550] – Holly M O’Neill

So I say experiment with different days and see what’s going to work best in your community. Thank you so much for listening and I look forward to hearing your questions.