Hello! We are Manolya Tanyu, senior researcher, and Nicholas Read, researcher at American Institutes for Research (AIR), a behavioral and social science research organization, sharing our insights on conducting virtual focus groups as part of a national evaluation of enhanced youth mentoring programs. We’re using videoconferencing as an alternative to in-person focus groups to save time and travel costs.
Lessons Learned: Virtual focus groups feel different than in-person focus groups, where the researcher is in the same room with participants and can observe group dynamics. In some of our focus groups, participants from the partner agencies met in one location and shared a computer with a webcam. This format allowed them to interact with each other but in most cases made it difficult for our remote facilitator to see everyone in the room. In other focus groups, participants joined separately via different computers. While this format allowed the facilitator to see all participants, it limited group interaction and participants tended to respond to the facilitator rather than participate in group discussion. In some instances, participants lost internet connection and were only present via teleconference, eliminating visual observation altogether. Overall, virtual focus groups can be an effective way to collect qualitative data, with a few parameters:
- all participants in the same room (if possible)
- reliable internet connectivity
- test the technology
- clear guidance to participants that focus groups should be group discussions.
Hot Tip: We used Cisco’s WebEx online conferencing (one of many options). WebEx allows participants to join via computer without having to download software, has both telephone and computer-based audio options, and allows video and audio recording. We obtained consent from each participant to be recorded (they signed and email us consent forms), recorded each session for transcription, and also had notetakers in case of technical glitches. Coordinating each virtual focus group took about five hours, including finding a date / time that worked for everyone, setting up WebEx, sending e-mail invitations, scheduling and conducting a test run, retrieving video and audio files from WebEx, sending audio for transcription, and cleaning up our notes.
Cool Trick: We highly recommend conducting test runs with participants before the actual focus group. For the majority of participants, this was their first time participating in an online focus group using videoconferencing. Test runs helped participants sort out technical issues and test audio and webcams. A 15-30-minute test run can save valuable time during the actual focus group.
Rad Resources:
- While focused on market research, Designing and conducting virtual focus groupshas more information on this topic.
- A little dated and a bit project-specific, Issues in Online Focus Groups: Lessons Learned from an Empirical Study of Peer-to-Peer Filesharing System Users also has good information.
The American Evaluation Association is celebrating Ed Eval TIG Week with our colleagues in the PK12 Educational Evaluation Topical Interest Group. The contributions all this week to aea365 come from our Ed Eval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.