Hello and kia ora, folks! I’m Jane Davidson, and I’m one of the pre-conference workshop facilitators at this year’s AEA conference in Washington DC.
Right now I just wrapped up Day 1 of a 2-day workshop called Actionable Evaluation: Getting succinct answers to the most important questions. Wow, we have great group in there, and they are brimming with really good questions!
Lessons Learned: Some of the stuff we talked about today:
- How to rewrite non-evaluative questions so they are explicitly evaluative (i.e., they ask not just “what happened?” but “was it any good?”). This was harder than it looked, I think, but we had some fun with it!
- My Key Evaluation Question “Cheat Sheet”, which workshop participants can tweak to create an easy set of high-level questions for their own evaluations. This will help them avoid getting lost in the details!
- How to develop evaluative rubrics, which you can use to interpret quantitative, qualitative, and mixed method evidence. Some fantastic follow-up questions for this.
Hot Tips: From today’s Q & A:
- Rubrics can be used for independent or participatory evaluation. If you go participatory, shoot for 3 to 10 participants; any more and it takes ages!
- When constructing rubrics, 4 to 6 levels of performance is usually about right. If you use too many you are imposing a level of precision that simply doesn’t exist in the underlying construct in the real world.
- Need to do a lit review but haven’t got the budget? Call 2 or 3 leading gurus on the topic and ask them for a quick live “brain dump” by phone or Skype. Cheap, quick, and credible!
Rad Resources: Related conference sessions:
- Evaluation-Specific Methodology – Wednesday all-day workshop (Michael Scriven & Jane Davidson) – or catch the 45-minute demo on the same topic at 11am Thursday!
- Rubrics: What, why, and how – Friday 8am panel session (Michael Scriven as discussant; Jane Davidson, Thomaz Chianca, and Monica Pinto presenting)
- Defining Which Side of the Elephant We Are On: Using Rubrics on Non-Profit and Government Program Evaluation (Kate McKegg & Juliet Dulles)
Rad Resource: Key resource for the workshop:
Actionable Evaluation Basics Jane’s e-minibook; also available in Spanish as Principios Básicos de la Evaluación para la Acción (translated by the awesome Pablo Rodriguez-Bilella) – coming soon in French (translated by Ghislain Arbour)!
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.