Greetings fellow evaluators! Our names are Veena Pankaj and Myia Welsh and we work for Innovation Network, a Washington DC-based evaluation firm. While Innovation Network has always used a participatory approach to evaluation, we recently came to the realization that much of the ‘participatory-ness’ of our evaluation projects was limited to evaluation planning and data collection. We suspected that an additional richness of context could be gained by including stakeholders in the analysis process.
We started by involving stakeholders in the analysis and interpretation of the data on a few projects. This helped us move from simply offering a final evaluation report with findings and recommendations, to embracing a practice that brought the client’s own perspective into the analysis.
Hot Tip: In determining whether participatory analysis may be a good fit for your evaluation needs, consider the following questions:
1. Quality: How might participatory analysis improve the quality of findings/recommendations?
2. Stakeholders: What might be the positive outcomes of engaging evaluation stakeholders?
3. Timeline & Resources: Will the participatory analysis approach fit within the project timeline and available resources?
Our experience in using this approach has helped us with the following:
- Present first drafts of data and/or findings, giving stakeholders the chance to provide context
and input on findings or recommendations;
- Help sustain stakeholder interest and engagement in the evaluation process;
- Identify which findings and recommendations are the most meaningful to stakeholders; and
- Increase the likelihood that findings and recommendations will be put to practical use.
Hot Tip: Conducting participatory analysis can be tricky. You are not just presenting ideas to stakeholders; you are facilitating a discussion process. Make sure you have an agenda in place, specific questions you’d like the stakeholders to consider and clearly communicated goals for the meeting. Having these items in place will allow you to focus on the richness of the discussion itself.
Rad Resource #1: Participatory Analysis: Expanding Stakeholder Involvement in Evaluation This recently released white paper examines the use of participatory analysis with three different organizations. Each example includes a description of purpose; the design, planning and implementation process; the effect on the overall evaluation; and lessons learned.
Rad Resource #2: Participatory Evaluation: How It Can Enhance Effectiveness and Credibility of Nonprofit Work For a different perspective, check out this article from the Nonprofit Quarterly. It discusses participatory evaluation practices in a community-based setting.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to firstname.lastname@example.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
3 thoughts on “Veena Pankaj and Myia Welsh on Participatory Analysis: Expanding Stakeholder Involvement in Evaluation”
Hello Veena and Myia from Vancouver, British Columbia. My name is Mary Hayward and I’m currently enrolled at Queen’s University in Ontario Canada doing my Masters in Education. The course I’m taking right now – Program Inquiry and Evaluation has opened my eyes to the world of programs and evaluations of their progress and growth on its participants.
A huge aha moment I had when reading articles by Dr. Michel Patton and Carol Weiss was the importance of involving stakeholders in evaluation process. Your article confirmed my findings that in order to increase the purpose and usefulness of evaluations stakeholder involvement would be beneficial. I really like your comment that when involving stakeholders in evaluations it will “increase the likelihood that findings and recommendations will be put to practical use”.
I’m currently creating a Program Evaluation Design where the biggest stakeholders of the program are my students themselves that would be going through the program designed to increase their social/emotional intelligence. Educators know that in order for students to show true lifelong growth they need to be involved in their own learning – thinking about how they’re doing learning a skill and ways that they can improve their learning.
Thank you for sharing and encouraging me to take the more challenging route of involving my stakeholders in program evaluations and have those rich discussions about progress towards the goals of our program in order to have true, authentic, purposeful and meaningful assessment.
Pingback: IE TIG Week: You’re Invited! How Data Parties Contribute to a Culture of Participation in Internal Evaluation Activities by Omoshalewa Bamkole – AEA365
I work for a small consultancy, Roberts Evaluation, in Melbourne Australia, we too promote participatory evaluation approaches amongst our clients.
One successful approach to engaging stakeholders in the analysis of data has been the Summit Workshop.
The Summit Workshop brings together a selection of the key stakeholders involved in the evaluation project, to whom the initial results and findings are presented – participants are then asked to draw out what they see as they findings.
The workshop aims to share lessons about the program, incorporate stakeholders’ experience and local knowledge into the analysis process and generate evaluation findings in a collaborative way.
We have used this method in evaluation projects across areas as diverse as state forest recreation management, bush-fire recovery programs, organisational change processes and weed and pest initiatives.
The technique was developed by Jess Dart at another Melbourne based consultancy, Clear Horizon, and their website offers a brief description, http://www.clearhorizon.com.au/evaluation-summit-workshop-technique/