I was catching up on my blog feeds recently and came across a post from Sam Ladner on her Design Research blog ( http://designresearch.wordpress.com ). A few months back she posted about the difference between qualitative and quantitative research as it relates to design research. The point of the post was to say that sample size is irrelevant in qualitative. Most smart researchers (like Sam) know this. So why does that basic tenet seem to disappear a bit when people think about an online research community?

First, allow me to explain…

We often get requests from clients who want to build large (on the order of 500+ people) online research communities. We first ask them about the type of output they’re looking for from their online research community. If their objectives lend themselves to primarily qualitative feedback, we generally recommend significantly reducing the size of the research community (on the order of 150 people).

This tradition for 300-500 member communities may have derived from early practitioners in the research community space, who recommended this community size as a means of getting a hybrid of qualitative and quantitative feedback. However, we’ve found that many client-side researchers and consumer insights specialists are looking for (and expecting) qualitative insights from research communities, so building a community in the 300-500 range is not always necessary.

Why smaller?

The reason goes back to Sam’s post: Sample size doesn’t matter in qualitative research. In fact, if you’re running a large research community and receive anything over 75-100 responses from individual community members to a given discussion or qualitative activity, you’re actually going to have a harder time building a sense of community and analyzing the feedback. Members and moderators will lose their attention span if they’re forced to read through too many responses, and you’ll lose a bit of the “across the table” discussion that can occur.

Does that mean my quant objectives are out the door?

Absolutely not. In fact, I had someone on a call the other day ask me if research communities were an acceptable venue for quantitative research. Given a large enough community (and therefore starting sample size) you can run quantitative research quite efficiently and effectively in an online research community.

So what’s the takeaway?

When you’re designing your approach to an online research community, consider keeping it small - particularly if your primary goal is qualitative feedback. The ideal is to receive between 30-50 responses to each discussion you launch. This will keep your feedback at a manageable level, while ensuring you get a wide range of opinions/attitudes on a given topic.

To learn a little more about the size debate, refer to our earlier post here: http://www.pluggedinco.com/2008/08/26/the-great-size-debate-no-not-that-great-size-debate/