Hello! I am Ravneet Tiwana, principal independent consultant with KHOJ Consulting. Evaluation questions are one of the most important elements in an evaluation design because they focus the study by functioning as flashlights in a deep sea of data. Often times, determining evaluation questions is something the client believes the evaluator should do as a hired expert. The evaluator often wants to partner with the client/key stakeholders to determine these questions in order to ensure usability of evaluation findings.
Lesson Learned: As evaluators, many of us know, that more engaged stakeholders lead to increased use of evaluation findings. The process of stakeholder engagement, according to Hallie Preskill Ph.D. and Nathalie Jones of FSG, increases quality, scope, and depth of questions, ensures transparency, facilitates the evaluation process, acknowledges the political context, builds evaluation capacity, and fosters relationships and collaboration. Ultimately, an evaluation supports evidence-based decision-making and brings transparency to this process when commissioned for accountability and/or continuous improvement purposes. Therefore, it is important to have good evaluation questions based on stakeholders’ primary information needs in order for the evaluation findings to be useful for these purposes. But, how do you do engage stakeholders? And do it, effectively? I found the included resources helpful in developing strategies to engage key stakeholders. One resource is helpful in identifying what kind of information is needed to create high-quality evaluation questions. The other resource is useful for identifying the most appropriate medium for engaging key stakeholders, particularly when you have quite a few scattered across cities, states, or countries.
Rad Resources: I have found the following resources helpful in developing evaluation questions with stakeholder groups.
- Evaluation Toolkit for Magnet School Programs: Provides a checklist for high-quality questions: Checklist: Developing High-Quality Evaluation Questions
- A Practical Guide for Engaging Stakeholders in Developing Evaluation Questions: Provides overall guidance on the purpose of good evaluation questions in ensuring useful, credible, and relevant evaluation findings. The guide also shares techniques and criteria for engaging stakeholders and the best medium for doing so.
Hope this helps in doing your work!
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hi Ravneet, thank you for the resources. I agree with you that evaluation questions are an important element in the design of an evaluation. I was however curious about the stakeholder participation in forming the questions. I believe that stakeholders should play an active role in evaluation design. once we have everyone involved, how do we unpack what is useable and what to leave in the periphery especially when there are opposing ideas on the whys of the evaluation.
Another thing that I wonder about is collaborative determination of evaluation questions. once we know what it is they want to find, do you ever find that you have to change their questions based on what they have told you the program goals are?
Yes, engaging stakeholders is critical to asking good evaluation questions. We created a free tip sheet with a flowchart for asking useful evaluation questions, available here: http://meaningfulevidence.com/publications