It seems like a lot of researchers and clients want to know how "research design" is different in a community than it is with point in time methods (like surveys, focus groups, and online focus groups). The answer can be slightly abstract. Community design is a bit more organic, adaptable, theme-based, and versatile, using a variety of tools at your disposal. In my opinion, it is not about using the same battery of questions, just posted in an online setting. It is about thinking differently about how you address topics and giving people new ways of expressing their opinions online.

Traditional Research Design

The easiest way to demonstrate this is with an example, so let's say you wanted to understand how teens use mobile phones. In a traditional or online bulletin board type focus group your question battery may be like this:

1. When was the last time you used your mobile phone? What did you do?
2. If you left home without your mobile phone, what would you miss most?
3. Tell me about how you use the phone feature on your mobile phone... what do you like about it? What don't you like?
4. Tell me how you use text... what are some of the challenges of using text on your phone right now?
5. How do you use the music feature... how would you improve it?

Online Community Research Design

• "Last Use" Discussion: Tell us about the last time you used your mobile phone... What exactly did you do?
• "Teens vs. Adults" Discussion: Let's say you were designing a mobile phone for Teens. How would your Teen Phone be different than typical mobile phones for adults? Think about the look, design, and the different features and functions you would want to include!
• "My Phone is Alive!" Photo Gallery: If your phone came to life as a person, animal, or another living thing - what would it be? Upload a photo of the "living thing" that best reflects your cell phone.
• "My Phone Diary" Blog Gallery: Keep a diary today and tomorrow of all the things you do with you mobile phone. So start with the moment you wake up this AM - and finish with the time you go to sleep tomorrow PM. Tell us every time you touch your phone... where you are, who you are with, what you are doing - everything!
• "Mobile Phones & You" Survey: Q1) What do you do most with your phone [call, text, listen to music, play games, use apps, other]? Q2) If you left home without your mobile phone, what would you miss most? [open end]  Q3) When you think about your cell phone, what 1-2 words come to mind? Q4-Q8) On a scale of 1-10, how important is the phone feature? [repeat for text, music, games, apps, etc.].

Organic Research Design

One important thing to keep in mind is that with the community environment, you have the ability to address objectives over time, and the community itself responds well to this more natural flow. You can take the findings/responses from this initial wave of research activity and incorporate it in your design of the next wave. In this example, it may be follow up discussions on the mobile phone features. You could identify the ones that members thought were the most important and run follow-up activities designed to understand how Teens are currently using these features and their thoughts about frustrations/gaps that these features need to address.

- Ben

I've blogged a bit before about the difference between a market research online community and an online research panel, but a recent discussion around the office sparked some more thoughts on the debate.  First, let me explain a bit of the background for this post...

Most of the community requests we get tend to be on the "smaller" side (i.e., sub 300 members).  These requests are generally from companies looking to supplement (or sometimes replace) their existing online and offline qualitative research through a MROC.  However, from time-to-time we get requests for very large (over 1,000 member) communities.  Upon receiving a recent request for a very large community, I started to wonder at what point a community starts to become a panel.  Allow me to explain...

What a MROC means to me...

I see the beauty of communities being the connections between members, and the resulting insights that occur through those connections.  I also see the benefit being a deep understanding of the individual and their needs, which is the goal of most qualitative research initiatives to begin with...  

However, when communities have many hundreds or thousands of members, those connections run the risk of breaking down.  Can you imagine the experience of a research participant who is asked to read through a discussion with 500 responses?  What is the likelihood of getting that valuable "across the table" discussion.  I know I wouldn't wade through more than 100 responses to a discussion if I was a research participant, even if I was very interested in the topic and shared a lot in common with other participants.  I just don't have the time...

In my experience, when a community exceeds about 300 members it starts to become a panel.  Connections between members don't happen as frequently, it becomes slightly less "personal," and it can become almost a Q&A session.  I don't mean to use the term "panel" in a derogatory manner, as there is always value in collecting feedback from a targeted group on a continuous basis - no matter the method.  However, I feel a true "community" should act and feel differently than a panel.

Comparing this experience to a focus group...

For anyone who has yet to experience or moderate a MROC, the only experience I can liken this to is if you were moderating a focus group and for each question in your guide you simply went around the table asking for answers in order.  Very large "communities" run the risk of feeling like a "serial interview," and less like a natural conversation.

This, of course, is just my two cents on the topic...  I can also see why some organizations want or need a larger research community; namely, that you can take the data from the qualitative discussions and validate it through surveys - all in the same venue.  The resulting cost and time efficiencies can be very enticing.  Plus, you can always take a very large community and sub-divide it into smaller groups of targeted people (given the right platform).  However, for my money I'd take a small group of committed participants any day :-)

What do you think?  When does a community become a panel?  What are the tradeoffs between a small and large community?  Comments are open (and welcome)!  

--Matt

I met a great moderator named Barry Zinger. He was a little frustrated after a recent round of focus groups. He had an interesting encounter with a professional research respondent and sent us the clip... check it out!

- Ben

One of the questions we're sometimes asked is whether or not it makes sense to recruit members for an online research community from a set of specific geographic areas.  The thinking is that later on it would be possible to meet with them in a traditional in-person research setting... 

This isn't only possible, it's smart and cost efficient.  It also points out a few different ways that online and offline qualitative research methods can be combined.  Read on for some ideas on combining online and offline approaches, as well as reasons why it makes sense to combine them...

How can you combine traditional with online qualitative research methods?

Below are some ideas for combining online and offline research methods to get you thinking about unique ways to design your next research study...

1. Recruiting community members to meet for in-person focus groups or in-depth interviews - This is the combination I referred to earlier, which consists of recruiting members from a specific geographic region(s) to an online community, and asking them during the screening process if they would be willing to participate in in-person sessions.  
2. Recruiting in-person focus group or IDI participants into an online community - Of course, the opposite to the first example holds true...  You could run multi-city focus groups and invite participants to continue the conversation in an online community afterward.  
3. Website usability sessions with an online community - Website usability is typically done in an in-person setting, most often as an interview.  Imagine what it would be like if you could invite them to connect online afterwards and take a look at the improvements you made to the site experience based on your feedback, while collaborating in a community setting to further optimize the site...    
4. In-person advisory boards with an online community or bulletin board focus group - Some companies have customer advisory boards that meet in-person at regular intervals (e.g., quarterly).  Why not give your advisors a regular place to meet online in-between sessions?  Alternatively, you could invite them to bulletin board focus group sessions at the mid-point between sessions...
5. Ethnography with an online community - Last (and definitely not least) is the idea of combining in-person ethnography with an online community.  Imagine having participants share diaries, photos, videos and ideas in between visits from the researcher.  That could lead to additional conversations and insights that would not have been possible with traditional ethnography alone.  
   

Why combine traditional with online qualitative research methods?

Thinking of combining online and offline qualitative methods?  Here are a few reasons why you should seriously consider this for your next study:

1. Cost efficiency - One of the biggest reasons for combining modes is potential cost efficiencies.  For example, recruiting members for an in-person study directly from your online community could help you save money on recruiting.  You might also be able to cut down on the number of in-person sessions you do, as you will be gathering quite a bit of feedback online as well.  Finally, you could also cut the number of markets for your study (thereby saving on travel costs), since the online component will help you cover a wide geographic area.  
2. Connecting between events - Ever wanted to follow-up with probing questions for that really insightful focus group participant, but couldn't because the group ran out of time?  Imagine having a community for connecting between in-person events.  The online community would ensure that you could follow-up with questions after analyzing the results from the in-person sessions.  That type of connection can lead to additional insights and ideas that would not have been possible by traditional research methods alone.
3. Seeing who participants really are - One of the major complaints against online market research methods is that you don't get to see the facial reactions from participants or generally see who they are.  However, combining online and offline research methods allows you to get the best of both worlds - an ongoing conversation through the online methods combined with getting to see them in a traditional research setting.  This might also help you screen out any "professionals" that might be in the midst of your studies...
4. Making homework assignments easier - Focus group moderators sometimes use homework assignments before groups to get participants thinking about the topic before they arrive.  Imagine how a community or online study before the group could coordinate members.  Also imagine what that might do to participation rates.  I would guess that more people will make the effort to show up for a focus group if there has been an online component before the group occurs.
5. Building rapport and getting started quickly - By getting to know participants online before a focus group, you can build up a baseline understanding of their background and start to develop rapport with members before they even step foot in the focus group room.  This helps you hit the ground running in the conversation, instead of spending valuable time just getting past introductory material. 
   

These are just a few examples of how traditional and online qualitative methods can be merged to create efficiencies and additional insights that would not have been possible with one approach alone... 

What do you think? 

Have you had positive or negative experiences with merging methods?  In your experience, which approaches work best?  Comments are welcome! 

Regular readers of this blog might be interested in a recent article on Mediapost.com called, "Listen Up, Marketers: The Focus Group is Dead."  Since it was posted this past Wednesday, 3/18, the article has sparked quite a few comments from market researchers, social media-types and fellow research community practitioners.  If you haven't checked it out already, I'd suggest giving it a quick read...  

The gist of the article is that brands are not actively listening to a broad base of consumers before making changes to their products, and instead are relying on traditional focus groups to make their decisions.  The article cites a few examples as supposed failures of focus groups, including Tropicana's packaging redesign, Motrin's recent ad debacle and even Facebook's redesign.  (I think these examples are a bit of a "stretch" to blame on focus groups, but hey...).  We've blogged about this topic a few times before, so I won't rehash our point of view.  If you have a few minutes to kill and are curious, you can check out our previous posts on focus groups.

What I thought was the biggest takeaway was not this sudden revelation that the traditional focus group is "dead" (which has been declared numerous times before, yet focus groups amazingly live on), but rather that it should be replaced by ways of actively listening through social media.  This is definitely a valid point...  However, even listening purely through social media has its drawbacks, as I started to allude to in my recent post on twitter as an online qualitative research methodology.  

The comment I felt was the most "balanced" came from Julie Schlack (of fellow community practitioners Communispace).  In her comment, she mentions that focus groups still have merit, private online communities are a great way to actively listen to concerns/ideas solicited from a broad base of customers, and how twitter is not the "be all end all" of listening.  I couldn't agree more with each of her points...  

What do you think?  Is this just more hype around the supposed "death of the focus group," or is the author of the article on to something here? 

Matt
(@mattpluggedin)

I'm a huge fan of The Onion and happened to come across an old article of theirs titled "Focus Group Reveals: 95 Percent Of Americans Would Like to Go Home."  It's funny and a bit disheartening at the same time, only because it's mostly true... 

Let's face it, the focus group participant experience hasn't changed much since this article was written (1997) and isn't likely to change much over the next 10 years.  You can doll up a focus group room with a "creative" atmosphere and feed participants all the M&M's they can stomach, but it's still the same experience (ignoring for just a second the role of a great focus group moderator and stimulating topic :) 

It's time for change in the qualitative market research world...

It's time that qualitative market researchers started to think more about how to enhance the experience of research participants.  I've recently come across a few demonstrations of new and unique survey question types that draw on "Web 2.0" tools to enhance the experience of survey takers, but haven't seen great strides in the qualitative research space.

The question is how qualitative researchers can make the experience actually "fun" for participants.  I'm reminded of a favorite quote of mine from Plato - "You can learn more about a person from one hour of play than from a year of conversation."  As qualitative researchers, how can we harness the concept of "serious play" into a better overall experience for participants and tangible research findings for our clients? 

Online qualitative research (including online bulletin boards, online focus groups, online research communities, etc...) has helped to change the focus group experience since this Onion article was written, by allowing people to participate at their convenience from home or work and offering up new ways for people to express their thoughts and opinions.  That's a step in the right direction.  I think qual researchers can do better though...

What do you think? 

How can qualitative researchers make research "fun?"  What role will online research play in this (if at all)?  At PluggedIN, this is something we're always thinking about in our research communities and research community platform, but we're always on the lookout for the latest and greatest.  Am I just not catching what's "cool" in the qualitative world?

Comments are welcome!

Last week I was in Philadelphia and Chicago for an in-person study with doctors and patients.  Given that we focus on research communities here at PluggedIN, this is something that I don’t get the opportunity to do as much anymore.  As much as I enjoy moderating online research communities, I appreciated the chance to get back out in the field conducting “traditional” methods again.

After the sessions were over I got to thinking - will online qualitative research methods (like research communities) ever replace traditional in-person methods?  If so, what would it take to “replace” traditional qualitative research methods (like focus groups, in-depth interviews and ethnography studies) with similar online methods?  If not, why not?  What are the barriers to replicating the offline experience?

Replicating the traditional research experience

The more I thought about it, the more I realized that the “traditional” research methods will likely never be replaced by online qualitative research.  As much as I’d like to believe in the future of online research (and online research communities in specific), the experience of speaking in-person with respondents just can’t be replicated by any currently available online methodology (that is, of course, until CNN releases their crazy Star Wars hologram technology to the world :)  Interviews with web-cams try to replicate the experience, but it’s just not the same as the “traditional” one-way mirror setup.

Instead, I see the future of qualitative market research as traditional methods being closely aligned with online methods to allow for a much deeper look into an audience over time.  For example, I foresee a day where participants at the end of every focus group are asked if they would like to join a corresponding online research community to further the discussion from the group.  I see companies and researchers collaborating with participants between in-person sessions using the tools provided in the research community.  This is happening now, but unfortunately only a few companies have realized the potential value of this online/offline relationship.

What do you think?

That’s my two cents…  What do you think?  Will online qualitative methods ever replace traditional methods?  If so, when will this happen?  What will drive this move to online qualitative research?

I just arrived back in the office from a day-long meeting with a client about the first set of findings from their online research community. The goal of the meeting was to present the initial set of community findings, understand what they mean for their organization and brainstorm ways to dig deeper in future months based on the feedback.

While marathon meetings like this can be a bit exhausting, it’s worth it when you hear stakeholders across all areas of the company (e.g., marketing, product development, customer satisfaction, etc…) react to the findings and realize that they now have this extremely valuable ongoing research community that can be used to gather insights much more efficiently and effectively than before…

What struck me as particularly interesting was a comment that came out in one of the early meetings. It was a comment made as we were providing some background on the community and what it was designed to do for their company. In this meeting a member of their core team drew the comparison of the research community to a “focus group on steroids.” I think that’s a great metaphor for an online research community, and here’s why…

Why this metaphor works

Here are some reasons why “a focus group on steroids” is a great way to think about an online research community…

• It draws on an existing schema - First, it uses an existing schema/concept (focus groups) that people know well, and applies it to a brand new methodology that can be tricky to immediately comprehend for anyone outside of a market research or consumer insights capacity.
• It sounds less “risky” - Similar to the above, a focus group is something that people know and are comfortable with. That increases their likelihood of accepting the findings as something that are valid and worth paying attention to.
• It sets the stage for the qualitative findings - It clearly identifies a research community as a qualitative research methodology designed to understand the “why” and “how,” and helps people understand that the community findings cannot be projected to a wider audience.
• It succinctly describes a benefit of an online research community - The “steroids” aspect clearly indicates that an online research community is superior to a focus group in many ways. This association can be made both in terms of quantity of feedback (which is almost always true of a research community) and quality (which is generally true, although focus groups can also generate valuable research insights in their own right).
• It’s memorable - It’s easy to remember and just “sticks” once you hear it. That’s a good thing when you’re hoping to draw attention to the availability of a research community across your company.

What does this mean for you?

Are you a researcher or consumer insights specialist who is having trouble “selling” the idea of an online research community internally? This might be a good metaphor to use to help convince your stakeholders that this is something worth pursuing. Of course, you can always elaborate on the metaphor by sharing some of the benefits of online research communities we share on this site.

Has this worked for you?

What do you think? Does this metaphor make sense? Have you used other analogies to help people quickly realize what a research community is and how valuable it can be? Feel free to leave a comment with other suggestions!