Greetings, my name is Giovanni Dazzo and I’m an evaluator at the Department of State and a doctoral candidate at George Mason University.
Over the last few years, I’ve been working alongside nonprofit staff and Kaqchikel community members in Guatemala, seeking to understand how applied research and evaluation can serve as a form of restorative practice for individuals who have experienced trauma.
This idea came after a day of participatory activities, where we co-defined our evaluation priorities and questions. As we shared a meal, one Kaqchikel community member said: “When you are asked to participate, but no one has ever asked you before, you’re afraid.” What was most striking about this statement was that this was not the first time that they had been engaged in research or evaluation.
Although many practitioners regard participatory methods as encouraging inclusion, we don’t often question the assumption that individuals feel comfortable participating in the first place. Do community members feel that they hold sufficient power and control to participate alongside us?
For this reason, the conceptual framework for this work begins with my Kaqchikel co-evaluator’s words. By reflecting on their words, I imagined approaches that could go beyond the usual talk of doing no harm; rather, moving toward the idea that evaluation can do good—in this case, by using it as a tool to promote peace, justice, and healing for those who have experienced trauma.
This also led me to question the validity of our discipline, but not in the typical way we think about the term. Rather, I’ve started to think about restorative validity:
- What does evaluation do for others?
- When we work with those who weoften define as marginalized, what does our evaluative work do to restore their humanity?
In reflecting on validity, I went back to its root, which simply means ‘strong’ or ‘healthy’. So, I questioned the strength and health behind evaluation when we ask communities to collaborate and participate, especially when their human rights have been violated by governments, corporations, and, at times, researchers/evaluators. In my work, it also means that one Kaqchikel community member no longer sees themself as afraid to participate.
- Recognize communities as co-creators of conceptual frameworks. Include co-evaluators’ words alongside those of prominent scholars, as they are likely influencing your thinking.
- Engage theory and practice. Urgency in practice can lead to dangerous assumptions. In developing this conceptual framework, these works—by Stanfield, and Chouinard and Boyce—have played a critical role in my thinking.
- Acknowledge those who welcome you. Before I end, it’s necessary that I recognize the time, knowledge, and space shared by Kaqchikel co-evaluators. This work is conducted on their land in San Juan Comalapa in Guatemala, near the site of Iximche’, the capital of their former kingdom.
The American Evaluation Association is celebrating International and Cross-Cultural (ICCE) TIG Week with our colleagues in the International and Cross-Cultural Topical Interest Group. The contributions all this week to aea365 come from our ICCE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to email@example.com. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.
1 thought on “ICCE TIG Week: Restorative Validity: Confronting barriers to participation and self-realization through participatory and collaborative moments in Guatemala by Giovanni Dazzo”
I was interested in your post as I am currently a student studying program evaluation design in a master’s program. I have a back ground working in the non-profit sector and am currently working at post-secondary institution as part-time faculty and in the disability services office.
Your post illustrated some interesting assumptions that we make regarding the communities within which we work. The underlying assumption that they want to and can participate in the process of evaluation. Your example of working with individuals who have experienced trauma is especially interesting because of the fact that there is a risk of retraumatizing someone by trying to measure outcomes regarding their trauma intervention. With that you mention “doing no harm”. When engaging participants as collaborators, how do you recognize the difference between harm versus healing to ensure the individuals are healthy and capable of participating? And at the same time, ensure validity if only the healthy participants can collaborate with you?
I ask these questions out of genuine interest regarding participant capabilities and then getting valid results. Working with individuals with disabilities, we need to measure outcomes to ensure our interventions are appropriate. However, when getting participant feedback, we run the risk of “creaming” when it comes to getting data. Only those capable of providing feedback do so. For a variety of reasons an individual may not be capable, healthy enough or finds the process stigmatizing and therefore won’t participate. From a social justice perspective how do we address this? Any insights here?