Hi. My name is Anna Douglas and I’m a PhD student in Evaluation, Measurement, and Applied Research Methods at Purdue University. I’ve been using asynchronous discussion groups online to evaluate a new learning tool.
Online asynchronous discussion groups are discussions through an online media where participants are responding to each other, but not simultaneously. While using these groups will not work for all evaluands, there are several situations in which the method makes sense. If you are interested in collecting qualitative data from a diverse group of individuals without being constrained by time or location, this evaluation tool is particularly useful.
Similar to traditional focus groups, participants respond to comments from other participants, and the evaluator can also ask follow-up questions. However, because the discussion is happening in an asynchronous manner, participants have time to think about the question and others’ responses before posting their response. They are able to develop their thoughts more fully, rather than responding immediately after the question is asked. The groups also provide an opportunity for participants to hear feedback from members who may be reluctant to share information in a face-to-face group setting.
One word of caution, while other disciplines have researched the use of online discussion groups (as an instructional tool), there does not appear to be much research on using these groups as part of the evaluative process. Until its efficacy as an evaluation tool has been established, it would be good to use asynchronous discussion groups as one source of evidence, not your only source.
Hot Tip: When setting up an asynchronous discussion group, it pays to work with the online media that is already in place, if possible. For example, most universities have online communication tools, such as WebCT and Blackboard. A webpage could also be established for participants, set up similarly to discussions on blogs. Social network sites could be utilized as well (isn’t Facebook just a lot of asynchronous discussions?!).
Hot Tip: Think about what rules should be in place for your evaluation needs and clearly state them for participants. For example, does it make sense for participants to have anonymity or be known? (there is some research indicating that anonymity brings out more bad behavior in online discussions, so participants may come across more strongly than if in person.) How long will you give participants to discuss? Should participants read each other’s posts and respond directly to each other, or just answer the question asked?
Rad Resource: If setting up a website for the discussion, Gravatar can be used to identify users on websites. The Gravatar website has tutorials on setting this up. The Genuine Evaluation blog utilizes this in their discussions as does aea365.
Anna will be presenting her work on using asynchronous discussion groups as part of the poster exhibition at AEA’s annual conference this November in San Antonio. Join us at Evaluation 2010 to connect further wish Anna and over 1000 other presenters!
Thanks, Anna, for this post!
This sounds like a very interesting method, particularly when, as mentioned, these sorts of discussions are already taking place.
I wanted to ask about the consent process – is there one typically used? if so, what would be included and how and when would it be administered?