Hello! We are Hanife Cakici, Christopher Pleasants, and Shipi Kankane from the Minnesota Evaluation Studies Institute (MESI) at the University of Minnesota. We were a part of the team of conference evaluators for Evaluation 2012 in Minneapolis. We asked conference participants to select a question from a menu, and videoed their responses to be shared with the rest of the evaluators during the conference. We were able to capture thirty responses and are excited to share our thoughts with you.
Lessons Learned:
- Love it or leave it: Most of the evaluators we approached agreed to be videoed. There were many others, of course, who were not comfortable being on camera. To comfort our participants, we put our “Minnesota Nice” hats, and acted gregariously by allowing them to use the front-facing camera as a mirror to fix their hair or tie.
- Privacy rules: We asked evaluators’ permission to share their videos on the AEA’s YouTube channel. Many did not concede. We assured them that the videos will be compiled in password protected, private channel that only the team has access to.
- Influx of Positive Responses: Videos are more likely to entice positive responses than other means of data collection. Evaluators often did not appear to publicly (or at all) share negative responses on camera. This ought to be taken into consideration when using videos as an interactive evaluation tool.
Hot Tips:
- God bless technology: iPads have been extremely helpful to take quality videos, and instantly upload them to YouTube. They are easy to use, easy to carry around, and super fun to play with! We are grateful for College of Education and Human Development at the University of Minnesota that allows students to check out iPads for free for up to two weeks.
- Evaluators in groups: We found out that we are less likely to be rejected when we approach evaluators in a group, which created pressure to respond to our “simple” questions. When evaluators were alone and especially eating something, we were most likely to be rejected. Make sure to understand evaluators’ ecology before you approach them.
- Adjust your YouTube settings: You can upload videos to YouTube with any of the followings settings:
- Public: Searchable to anyone on YouTube
- Unlisted: People with URL can view video.
- Private: Sign-in required viewing videos.
- Make yourself visible: Conference participants needed to know ahead of time who we were and what we wanted to accomplish with our short video-questionnaire so that they would entrust their responses and images to us.
- Have fun: Check out our Evaluation 2012 Video! Thank you all for your contributions!
We’re learning all this week from the University of Minnesota Innovative Evaluation Team from Evaluation 2012. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluator.
Thanks Team! I always like to hear from attendees – although I share the concerns raised about the overly positive tone. I’d be intrigued to learn more about how you chose which voices to highlight from among those recorded.
Nice observations. After reading through, wondering if it might be worthwhile further exporing your third point re the skew toward positive responses from those agreeing to be taped.
Since folks are justifiably leery of publicly critiquing their peers or AEA, questions requiring pro/con opinions don’t seem ideal for video. Still, the medium may work well for descriptive or fact-based questions, such as your q’s on what brought people to the conference and their big aha’s. Video might also be good in soliciting suggestions, for example asking first timers how AEA could help them better connect with people. Cheers, josh