Hello! We are Jessica Robles, Nitya Venkateswaran, and Jay Feldman from RTI International’s Center for Evaluation and the Study of Educational Equity. Our team was excited to see that AEA was soliciting blog posts about the challenges of sharing truth through evaluation in the nonprofit sector, as it would give us an opportunity to reflect on our collective experiences over the years and share lessons learned.
One question we always ask ourselves at the beginning of an evaluation is, “How do we make sure we are using participants’ voices to tell their truths?” And in some cases, “How do we make sure we are telling participants truths, especially in the face of inequitable power or race dynamics or when a gatekeeper wants to alter or own those truths (intentionally or unintentionally)?” Here are key lessons learned for consideration, especially for evaluations serving people from historically underrepresented groups:
Lesson Learned: Ensure a representative group of stakeholders (e.g., students and parents) is included from the onset of the evaluation and build an appropriate structure for their involvement. Hearing from a broad range of stakeholders throughout the process has given us consistent “check points” to make sure the evaluation is representing everyone fairly.
Hot Tip: Some program leads are not used to soliciting feedback from anyone beyond their core team or organization. If this is the case, explain the benefits of doing so, and provide them with resources to support them as they learn about this practice (such as this action guide or Cousins’ and Earl’s classic, The Case for Participatory Evaluation).
Hot Tip: The evaluation plan should clearly state that stakeholders will respond to drafts and the timeline should prioritize soliciting and integrating their comments as way to lift up multiple voices. Making their inclusion a non-negotiable prevents potential gatekeepers from removing uncomfortable or negative findings. Feedback from different groups, even (especially!) when extensive, helps ensure participants’ truths are told accurately and strengthens buy-in to the findings.
Hot Tip: As evaluators, sometimes our work is negotiated with a single point of contact. If a program lead is not responsive to the need to include stakeholders, find a champion who works closely with them to advocate for this.
Lesson Learned: Always use culturally competent ways of engaging stakeholders. We cannot assume everyone is comfortable providing feedback on the phone, in a survey, or using track changes in Word. We found it was critical to learn about what works best for stakeholders to ensure they have a fair chance to give feedback (e.g., offer meeting times outside of 9-5, consider translators, etc.).
The American Evaluation Association is celebrating Nonprofits and Foundations Topical Interest Group (NPFTIG) Week. The contributions all this week to aea365 come from our NPFTIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.