Hi there! My name is Lindsay Anderson. I am a PhD student at the University of Minnesota studying Organizational Leadership and Policy Development with an emphasis in Evaluation Studies. Having worked in social work before returning to school, I hold a high value on the importance of relationships and the notion that working together leads to better results.
Collaborative evaluations actively engage program stakeholders throughout the evaluation process and include approaches such as participatory (shared control), empowerment (stakeholder control) and collaborative (evaluator control).
Involving stakeholders can result in many benefits to an evaluation including increasing quality, effectiveness, ethical alignment, utility and use. Collaboration may help increase stakeholder understanding of the evaluation purpose, improve data collection and reporting quality, increase access to program resources, further the dissemination of evaluation results and facilitate program change.
Hot Tip: Identify WHO potential stakeholders are.
Stakeholders are anyone with a vested interest in the program and who therefore also have a stake in the evaluation.
- Program participants may provide first-hand experience of the program being evaluated and are the most likely to be impacted by the program and evaluation.
- Partnering organizations and community agencies can provide insight into the context in which the program is embedded.
- Program providers represent multiple perspectives within the organization and build understanding of program activities and outcomes.
- Primary users of the evaluation are instrumental in implementing evaluation findings.
Hot Tip: Decide HOW stakeholders will be involved.
Formal strategies to involve stakeholders in an evaluation can include forming an evaluation advisory group or conducting one-on-one interviews and/or focus groups. An evaluation advisory group consists of stakeholders and evaluators that meet regularly throughout an evaluation to discuss evaluation materials and progress. Interviews or focus groups do not meet with regularity but can be useful in gathering ideas to define and revise evaluation plans.
Hot Tip: Decide WHEN stakeholders will be involved throughout the evaluation.
Stakeholders can be involved throughout the entire evaluation process.
- Clarifying the evaluation plan: stakeholder perspectives provide information about program activities and expected outcomes to ensure the evaluation purpose and design align with program functions.
- Data collection: stakeholders can be engaged to refine data collection strategies to maximize participant response. Evaluation instruments may be designed and validated through consultation with program experts and pre-existing program datasets can be utilized for data collection.
- Data analysis: stakeholders can provide their interpretation of analyses, offering another perspective to triangulate findings and improve the accuracy of results.
- Reporting findings: stakeholders can improve reporting of findings by: providing feedback on the mode in which results will be shared; ensuring reports are user-friendly; and expanding networks so results reach a larger audience.
Rad Resources:
- Campos-Rodriguez, L. Collaborative evaluation. Tamarac, FL: Lumina Press.
- Campos-Rodriguez, L. (2012). Stakeholder involvement in evaluation: Three decades of the American journal of evaluation. Journal of multidisciplinary evaluation, 8(17), p57-79.
- Greene, J.C. (2005). Stakeholder. In S. Mathison (Ed.), Encyclopedia of evaluation. Thousand Oaks, CA: Sage.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hi Lindsay,
My name is Sarah and I am currently an elementary teacher in Alberta, Canada and completing my Masters in Education with a focus on Assessment and Evaluation. The AEA365 site was introduced to me through a Program Inquiry & Evaluation course, and I have been enjoying the collaborative and applicable aspect that the site has provided when learning about evaluation models. Your article caught my interest due to its connection to our current learning about types of evaluations and associated dilemmas.
When examining participatory and collaborative evaluation, I found that part of the dilemma with these inclusive approaches is selecting the right “who” and “how” these stakeholders should be involved in order to gain fair and valid insights for the evaluation. Another dilemma I highlighted was “when” to involve everyone, knowing just how important stakeholder’s contributions are, but also understanding the consuming role of the evaluator. I found your article to be a clear and succinct road map to these dilemmas, specifically the concept of triangulating findings with stakeholders.
I appreciated your guiding descriptions to help me understand that collaborative evaluations can be manageable. In particular, your Hot Tip on “when” to involve stakeholders has assisted me with my final assignment of designing a program evaluation for a social program of our choice. Since my selected program involves many stakeholders ranging from students to corporate funders, your descriptions of involvement has helped me structure that aspect of my evaluation design.
Many thanks for your contribution to AEA365!
Sarah Walker
Dear Ms. Anderson,
I really enjoyed reading your article, Stakeholder Involvement and Collaboration I Evaluation, as it clearly outlined the three approaches for engaging stakeholders in Collaborative Evaluations. I was keen to read your article, as I am currently enrolled in a Master’s of Education course regarding Program Evaluation and collaboration is an area that interests me.
I loved that you began by stating the importance of relationships! As a teacher, building relationships with my students and their families has helped to support their learning. Fostering relationship capacity is one of the Alberta Education Teaching Quality Standards.
While reading about the benefits of including stakeholders in evaluation, I also thought about how these align with the Program Evaluation Standards. When I read one of your Rad Resources, Stakeholder involvement in evaluation: Three decades of the American journal of evaluation, it outlined the evolution of the different approaches regarding stakeholder engagement. By going back and rereading your three Hot Tips of the WHO, HOW and WHEN, it clarified the stakeholder involvement during the evaluation process. These Hot Tips reminded me of the Collaborative Response Model, that I have been involved with for the past year. The ideas you have outlined aligns with how the Collaborate Response Model aims to engage with stakeholders.
Thank you for emphasizing how valuable it is to collaborate with stakeholders during the evaluation process.
Kind regards,
~Crystal
Hi Lindsay. My name is Maya Milne and I am currently completely a course in Program Inquiry and Evaluation as part of my Professional Masters of Education at Queen’s University. Our course has recently discussed the topic of collaborative evaluation and I was immediately intrigued by the topic. In my current profession as a Registered Dental Hygienist, my role in evaluation is more likely to be that of the stakeholder. The concept of collaborative evaluation provided me with a greater vested interest in the subject of evaluation on a professional level, I could relate to it personally. I appreciate how you broke down collaborative evaluation into three approaches, almost a gradient in the amount of participation each party has in the evaluation. I could understand how these approaches could be applied in different evaluations. In your Hot Tip, the WHO, HOW and WHEN really clarified the process of deciding the level of stakeholder involvement in the evaluation. From my research I have begun to see collaborative evaluation as a mutually beneficial arrangement where having both parties (evaluator and stakeholder) involved in the evaluation will benefit them individually as well as the evaluation process in the end. Thank you for your role in furthering my knowledge of evaluation.
Hello Lindsay:
Thank you for your article on Stakeholder Involvement and Collaboration in Evaluation.
I am very interested in collaborative evaluation and see the benefits as described in your posting. Particularly that participants have first-hand knowledge of the program, and that they can provide both a history and a context of the program. Coupled with the stakeholder’s interest in the program, is as you indicate, “…a stake in the evaluation.”
I was wondering what your thoughts of collaborative evaluation are with respect to misuse of the evaluation. Shula and Cousins (1997) state that, “…the swing toward collaborative models raises serious questions about the evaluator’s ability to maintain a sufficiently bias-free stance due to pressures emanating mainly from within the program community.”
Do you agree/disagree with this statement? If you agree with the statement, do you think that evaluators would be better able to assess situations of misuse through consultation of AEA’s, Guiding Principles for Evaluators and its five guiding principles, “systematic inquiry, competence, integrity and honesty, respect for people and responsibility for general public welfare?”
Thank you for your time and I look forward to having an engaging dialogue with you.
Best regards,
Angela
Thank you so much Lindsay! I found this article to be meaningful, straightforward, and with a refreshing tone that made it easy to read and comprehend. I’ll be sure to check out those ‘rad’ resources!
My name is Mason Foulkes, and I am a teacher in Coquitlam BC, Canada. Currently I am in a Masters Program course on program evaluation, and I am fascinated by the idea of stakeholder involvement.
One one hand, I see the value in program evaluation. It seems logical that we need to review what we are doing to see if it works, especially in the field of education. On the other, I see the day to day ‘I don’t have time for this!’ that teachers are constantly bombarded with. And while I don’t necessarily agree with that stance, I can’t discredit it on account of so many of my peers live by it.
My train of thought is leading me to these questions. What can a program evaluator do to make stakeholders feel that their time and input is important, when a lot of the time people won’t even give the preamble or plan the time of day to review? Similarly, what can stakeholders do to empower each other, and how do we spread the collaborative spirit to encourage more meaningful stakeholder engagement?
I see so many people brush off program evaluation and dismiss their status as an important stakeholder, right off the start. Should effective program evaluation spend time on empowering stakeholders? Or should it try to invite them as much as possible, while still focusing on the goals of the evaluation. How important is meaningful stakeholder feedback to effective program evaluation?
Thanks so much! Loved the read!
Cheers!
Mason Foulkes