My name is Kim Norris and I am the Evaluation Coordinator for University of Maryland Extension’s Food Supplement Nutrition Education (FSNE) program. Included in my work is to assist educators in developing useful strategies for assessing the impact of their work on our target audience, limited-income, and often, low-literacy, populations.
Hot Tip: Utilize Audience Response Technology System in group class settings for immediate, anonymous assessment and feedback. The “clicker” technology, as we sometimes refer to it, allows questions to be asked both orally and in writing, allows individuals to respond anonymously, and allows for immediate feedback after responses for both educator and audience. We used these recently with our own educators to ask questions that, in a setting in which anonymity was not guaranteed, could lead to false answers due to high motivation to fall within socially acceptable norms. Since results are calculated and visible to all on the spot, group responses can be reviewed, analyzed, interpreted, and addressed by the group, thereby increasing potential for empowering participants.
Other advantages of using this easy-to-teach technology include the ability to:
- collect data from larger numbers of people in a shorter amount of time
- eliminate data entry errors by direct transfer of electronic data to a data base
- engage technology-averse populations in computer technology to their benefit
- help low-literacy populations participate in surveys as respondents
- provide confidentiality for respondents
The technology can lead to missing data if not preceded by sample questions and as group sizes become larger or less engaged. Studies are underway to better understand strengths and limitations of the technology as an educational and evaluation tool.
Rad Resource: A Bibliography of Selected Readings on Audience Response Systems: http://bit.ly/audienceresponsesystems.
This week’s posts are sponsored by AEA’s Collaborative, Participatory, and Empowerment Evaluation Topical Interest Group (http://comm.eval.org/EVAL/cpetig/Home/Default.aspx) as part of the CPE TIG Focus Week. Check out AEA’s Headlines and Resources entries (http://eval.org/aeaweb.asp) this week for other highlights from and for those conducting Collaborative, Participatory, and Empowerment Evaluations.
Hi Kim!
This was such a useful post on audience response. I believe that audience response is important and the feedback you can receive from the audience can be very impactful. This type of feedback an aid in so many areas whether it be the audience itself or just the actual feedback from the program as well. I love the tips that you mentioned in this post. The clicker technology is definitely a very useful tool in many different aspects of audience response. Great post!
This is very useful tool.
David and Marcus,
Thanks for the comments. University of Maryland Extension is investing in this technology for their field offices for education and evaluation purposes. It is really catching on! We have had one eductaor conduct a small validation study with favorable results.
Kim
I agree with you 100%. I love the “The “clicker” technology.”
We have used it in the Medical School here at Stanford University to get a quick read on the medical students’ views of the curriculum (during a pizza lunch focus group – between classes). It is also a great way to launch a focus group – starting with the group’s views of what’s going on. They love to see the bar charts that are created automatically – showing them where they stand – in the mainstream or as an outlier. It is a great way of letting decision maker’s know what the relative proportion of positives and negatives are (which is hard to say sometimes in a focus group). Thanks for mentioning this tool – it is very cool indeed (and costs are coming down a lot – just have to make sure everyone turns in their clicker at the end of the session). Best wishes.
– David
Hi Kim,
Great technology indeed. A few years back, I helped with an eval of one public high school–supplementing focus groups with the audience response technology. I strongly feel as though if we did not use such technology, social desirability definitely would have been an issue (as noted by some teachers’ reluctancy to become fully engaged and lack of openness during the focus groups).