Hello, this is Hannah Appelbaum, Brittany Edwards, Andy Grewell, and Jessamay Thompson. We are the team of graduate students that evaluated the Ignite talks (http://igniteshow.com/) at this year’s AEA conference. Because they were a new feature this year, we chose to evaluate these talks in order to assess their effectiveness for inclusion in future conferences.
What is Ignite?
Ignite talks are fast-paced, thought provoking lectures lasting only 5 minutes and containing just 20 PowerPoint slides (that’s just 15 seconds per slide, folks). Ignite now has a network of events and speakers embracing this innovative approach. According to the Ignite website, Ignite is now a series of “geek events in over 100 cities worldwide.” Videos for Ignite talks are available on YouTube and the AEA Ignite talks will be posted there as well later this year.
To match these high-tempo lectures, we created a survey with just 5 questions, and lasting only 20 seconds. Here is how we did it:
Rad Resource – Quicktap Survey: We sought a resource that met the following criteria:
- Get rapid reactions
- Elicit as much feedback as possible
- Mobile and “on-the-spot”
- Not require ongoing internet access (the Convention Center had spotty wifi access)
We stumbled across QuicktapSurvey, a free app available for iPads that allows respondents to hold the iPad and simply tap their answers quickly.
Here are a few examples of what this looked like for the respondent.
Our survey questions were the following:
- How many years of experience do you have as an evaluator?
- The ignite session met my expectations.
- This ignite session was ______________. (Select all that apply.)
- How likely are you to watch future AEA Ignite talks?
- Would you recommend attending Ignite talks to a friend or colleague?
And this is what we found:
- 84% of the 155 respondents either strongly agreed or agreed that the Ignite session they attended met their expectations.
- Only 3% of respondents reported they were not likely to attend another Ignite session.
- 95% of respondents reported they would recommend Ignite sessions to a friend or colleague!
Hot Tips
- Limit the number of questions
- Take advantage of the different question formats, we liked the simple “Yes and No” question, which elicits an either/or rather than neutral responses, and the “sliding scale” format.
- Catch people who leave early–stand in the back of the room or in the hallway.
- Have the moderator plug the evaluation either before or in between presenters.
- Have fun!
We’re learning all this week from the University of Minnesota Innovative Evaluation Team from Evaluation 2012. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluator.