Happy Saturday! My name is Elizabeth DiLuzio, AEA365 Curator + Contributor and Evaluation + Strategy Manager at Good Shepherd Services. I don’t know about you, but I’ve been engaged in a lot of conversations about participatory action evaluation (herewithin participatory evaluation) lately. It seems as if we are all in our corners of the country, processing the daily cries for social justice and reflecting on how we can better contribute to the movement. With today’s blog, I want to start an even larger conversation in our community: how do you see participatory evaluation contributing to the social justice movement? What are some of your favorite ways to incorporate participatory methods into the evaluation process? And what are some of your favorite resources? To follow, I’d like to share some of my own reflections on these questions. If you would like to share your thoughts, check out the end of this blog for how you can join in the conversation.
The Value of Participatory Evaluation
There are a number of benefits that a participatory evaluation approach offers. To me, the most important is the empowerment of the involved communities. Done well, a participatory evaluation enables them to learn theories, mindsets, frameworks and skills from the evaluators involved. At its best, a community-led participatory evaluation offers the opportunity for its members to ask questions about the world around them and take informed action based on their learnings. I’ve been using this approach since I was an 8th grade teacher, empowering my students to identify and explore their passions through targeted inquiry. The unit taught them skills such as writing, math, history and citizenship and was always one of their favorites because they were in the driver’s seat.
I also appreciate how participatory evaluation turns traditional research on its head. Our country has a long and fraught history of abuse of vulnerable communities at the hands of researchers. While examples such as the Tuskegee Experiments and genetic research of the Havasupai tribe are often provided as examples, the list of offenses is embarrassingly long. This has created mistrust, lack of engagement, and at times outright hostility towards researchers. Participatory evaluation done well, however, challenges the purpose of research and posits that the learning should be for the advancement of the community from which the information comes. It centers itself on providing equal or greater value to the community than it does to the evaluator. It creates a fertile ground where trust and relationships can grow.
Reflecting on Your Participatory Evaluation Style
If all of this sounds fabulous to you, you may be wondering how to engage in participatory evaluation. Or, if you already do, you may be wondering if there are ways to do the work even better. In the rest of the blog, I’d like to take a look at a couple of frameworks for conceptualizing and reflecting on participatory evaluation practices.
But first, a distinction: there is a difference between participatory methods and participatory evaluation. Participatory methods are the ways to approach evaluation work. They are creative structures that engage people in decision-making. The more frequently participatory methods are successfully utilized throughout the evaluation, the closer the evaluation gets to being participatory. At its fullest potential, it’s the community that takes the lead in every stage of the evaluation, offering participation to the evaluators when their expertise is desired. This could be envisioned on a continuum, such as in Figure 1.
While the fully realized version of participatory evaluation is not feasible for every evaluation, there are always opportunities to get closer to the ideal. In order to conceptualize how this might look, we could consider an evaluation project to be broken down into five distinct phases in which decisions are made (Figure 2). By taking a critical look at whose voices are influencing and ultimately making decisions at each of the phases, we can begin to identify areas for growth towards a more participatory approach.
Being a Participatory Evaluator
Another important ingredient to a successful participatory evaluation is having the proper mindset and skills. For example, any evaluator interested in conducting participatory evaluations needs to be self-aware about his or her privileges and biases. Without a deep understanding of both, it is easy to undermine the values and intentions behind participatory evaluation.
Participatory evaluators also benefit from strong facilitation skills. In particular, they are able to build trust, build relationships, create a safe and welcoming environment, solicit feedback from everyone in the room, summarize discussions in a useful ways, be attentive to the needs of the group, be able to change plans and improvise as needed, keep meetings engaging, and have confidence and staying power when things get difficult (SEI, 2008).
Rad Resource: For more ways that you as an evaluator can show up in a truly participatory fashion, check out “Why am I Always Being Researched?” by Chicago Beyond.
The Benefits of Participatory Evaluation
The quality of the evaluation improves as the evaluation nears a fully participatory approach, from the resulting data, the quality of life of the community, and the relationship between the community and the evaluators. You can find concrete examples of the improvements realized from incorporating participatory practices at each of the stages of evaluation in Figure 3, below.
Rad Resource: Click here for a copy of Evaluation + Learning Consulting’s guidebook of participatory evaluation activities adapted to the remote environment.
Now it’s your turn: How has your use of participatory evaluation contributed to the social justice movement? What are some of your favorite ways to incorporate participatory methods into the evaluation process? What are some of your favorite resources? Share with us in the comments below or in our Evaluators’ Slack Channel, where you can comment, share links, and even upload resources. It’s easy to join and free to use. We’ll see you there!
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
I haven’t personally utilized a participatory evaluation approach, program evaluation is new for me. I am currently taking a course on program evaluation design. There are many concepts within your post that I found very interesting. I am in the process of developing a program evaluation design where I am utilizing a comparative case study approach but wanted to include a participatory approach as well. I am seeking to evaluate a program that has been designed to increase graduation rates among First Nation students. In your post you mention “empowering the involved communities”. As an Anishinaabe kwe, I have come to recognize that our people have been under the microscope since the beginning of contact. I couldn’t agree more with your statement, “Our country has a long and fraught history of abuse of vulnerable communities at the hands of researcher”. I love the idea of including participants in the evaluative process. Weiss (1998) suggests that it helps to readdress the inequalities in access to influence and the imbalance of power that beset marginalized groups in their social interactions with larger society” (p.30). I am able to understand and recognize the factors that have contributed to the lack of trust between Indigenous people and non Indigenous people and the many inequities that Indigenous people face in Western Society. I think its important to think about who’s voices are part of the dialogue and decision making. I also think that the relationship piece is vital particularly when working within our First Nation Communities. Taking the time to build trusting relationships will prove beneficial in the long haul. I am grateful for the opportunity to learn more about participatory evaluation approach. As an advocate for my people, I know that building my own knowledge and understanding of evaluation can only benefit the First Nation communities that I work with. Miigwech (thank you) for sharing your knowledge 🙂
Ramona
I love that diagram (figure 1)! It really helped to put framing and words on my evaluation approach aiming to be more cooperative and co-learning focused, to offer as contrast to the more traditional models. I’m even going to use in a proposal immediately. Thank you!
Hi Elizabeth, thanks for this interesting post. The continuum you’ve outlined (Fig 1) is particularly useful for understanding how to move evaluations to a more participatory style. In one of the programs I implement in a refugee camp context, we have a challenge with implementing participatory approaches because the population is so “over-researched.”
Dozens and dozens of research studies are conducted every year (often sampling some of the same people over and over), so interest in participating in studies and research activities gets lower and lower by the year! I understand that participatory evaluation is about letting participants set the agenda so that it can be meaningful to them…but are there any best practices out there about how to begin this engagement with a population that is, simply put, tired of being researched? In this scenario, is a participatory approach even worth it unless we can be confident that any results/recommendations will be truly followed through on? Otherwise, we risk further discouraging participants and reducing their interest in this type of engagement.
Yes, Uma Kothari discusses this in Participation: The New Tyranny .
Approaches and methods are sometimes perceived/ portrayed as panaceas for the power differential between the knower and the known, but are not necessarily–as long as the questions are not rooted in and arising from the community’s own interests/ work. Even in community based participatory action research, the relationship with “community” is typically initiated by a faculty member and serves their professional interests.
I wonder what evaluation would look like if the refugees were the ones asking the questions and resourced to hold those organizing the camps accountable?
How is participatory evaluation like or unlike action research?
Good morning, Margaret. That’s a good question and I think the two are more or less the same. They have the same principles, values and approaches, although the intended purpose is a bit different. While CBPAR, PAR or action research all intended to involve the community in learning about its environment and present challenges, participatory evaluation seeks to do this in the context of evaluating an intervention or program. There’s debate as to whether research or evaluation is the broader “umbrella” concept, or whether they are similar or overlapping concepts (https://www.betterevaluation.org/en/blog/framing_the_difference_between_research_and_evaluation), but ultimately the two are more similar than not.
I hope that was helpful!