Hi there fellow evaluators. We’re Peter Rudiak-Gould and Rochelle Zorzi, evaluation consultants (and obsessive seekers of better ways to do things) at Cathexis Consulting in Toronto, Canada. We’d like to share a simple trick that makes Developmental Evaluation much easier and more productive.
Developmental Evaluation sounds easy. Traditional evaluation is an exhaustively rehearsed dance routine, while Developmental Evaluation is just moving your body to the music. But when you try it, you may find yourself wishing for a choreographer! It’s hard to explain to a client exactly what Developmental Evaluation is. You can’t make any promises about the questions that will be answered or even the methods that will be used, because that will change and grow along with the program. You can’t even tell the program staff how much of their time the process will take. All of this may leave your client (and yourself!) asking: “Wait…what are we doing again?”
We have found that one simple technique can go a long way in bringing some method to the madness. We call it the “Burning Question.”
Cool Trick: Each month, focus the Developmental Evaluation process on a single “Burning Question,” formulated by the program staff with some coaching by the evaluators. The Burning Question is the single most important thing that the program staff need to figure out that month, in order to design or run the program effectively.
Hot Tip: We recommend actually using the phrase “Burning Question,” because it communicates a sense of urgency and 
relevance.
Once the program staff have decided on a Burning Question for the month, help them to identify some very easy data collection that they can do within the month to begin to answer it. At the next monthly meeting, reflect on what’s been learned about the previous month’s Burning Question through that data collection. Then decide on the next month’s Burning Question and how it will be answered. (Next month’s Burning Question can be the same as the previous month’s Burning Question, if it still feels urgent and needs further investigation).
Lesson learned: The Burning Question technique gives a bit of structure and predictability to the Developmental Evaluation process – program staff won’t know what question they’ll be tackling each month, but at least they know that there’ll be a question! The technique also makes sure that the process focuses on information needs that really matter right now, rather than data collection for its own sake. As trust in the process builds, program staff can start to tackle more ambitious evaluation activities: bigger questions with longer-term data gathering, KPIs, a dashboard…But they won’t do any of this until they’ve gotten a flavor of evidence-based decision-making – and that’s where the Burning Question shines.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Pingback: Episode 11: A Co-Created Conversation – Eval Cafe