Hello, AEA365 community! Liz DiLuzio here, Lead Curator of the blog. This week is Individuals Week, which means we take a break from our themed weeks and spotlight the Hot Tips, Cool Tricks, Rad Resources and Lessons Learned from any evaluator interested in sharing. Would you like to contribute to future individuals weeks? Email me at AEA365@eval.org with an idea or a draft and we will make it happen.
Hi there! My name is Kiani. I am an evaluator at Evaluation + Learning Consulting and a Social Welfare PhD student at The Graduate Center, City University of New York (CUNY). I recently completed an evaluation of an innovative program funded by New York Community Trust (NYCT) and developed by Alexandra Martinez, the Director of Adult Services at Kingsbridge Heights Community Center (KHCC) in the Bronx, NY. This exciting program provides support to bilingual social work students at Lehman College during an academic year; consisting of monthly coaching calls in Spanish, six culturally responsive seminars, and a final presentation.
The NYCT project was my first experience completing an evaluation for a program that connected to a specific personal experience. As a Latina born and raised in Miami, FL, speaking Spanish was an expectation. Many of my family members did not speak English. Thus, I spent much of my childhood translating mail, official documents, and applications (e.g., employment, social services). As I entered the professional space, the exposure to Spanish lessened and it became difficult to articulate my work, passions, and goals to my family members.
As evaluators, one of our expectations is to be disconnected from the work so that, when we evaluate programs or organizations, our feedback is inevitably “un-biased.” However, it is nearly impossible to remove all the biases we acquire from our lived experience, and it can be a huge disservice to do so. Evaluating the NYCT project is evidence of how lived experiences can strengthen our work instead of hindering it. For this to hold true, there are factors to actively address to maintain the integrity of an evaluation. I write this article to share my experience and lessons learned when evaluating a project deeply connected to personal experience for the first time.
You can imagine my excitement when approached with evaluating a project that provided support to bilingual social workers. It related to personal challenges that I’d experienced entering the professional realm: my inadequacy in communicating professionally in Spanish. And, since I was invested in its impact personally, I was uniquely critical of the program’s components as someone who deeply understood the potential needs of a bilingual worker. As such, focus groups and interviews conducted with participants were fruitful as I leaned on my personal experience to ask specific follow-up questions and “dig deeper.” My experience as a native Spanish-speaker was also an asset to the evaluation process; since I entered the space with context and insight, participants shared feedback more openly.
To establish integrity and avoid an oversaturation of personal experience pouring into the analysis, I intentionally considered two factors:
- Let the data speak for itself. Our lived experience may prompt initial ideas about the usefulness or potential impact of a project. We should delineate between our ideas and the ideas that are being drawn from the evaluation.
- Immerse yourself in the program (to the extent that you’re able). Taking the time to truly understand the program components and its connection to the project’s overarching goals is critical to a successful evaluation and credibility. It’s particularly important when the project connects to an evaluator’s personal experience because it enables the evaluator to focus on the story the data is telling rather than relying on any assumptions from preconceived notions.
I fully support the push to uplift participant voice in research and evaluation. As researchers and evaluators, it’s equally important to consider how our lived experiences influence our processes and interpretations. Whether it is additive, like the example I shared here, or reductive; introspection is critical to completing a holistic and realistically “un-biased” evaluation.
Rad Resources
- Barrows, A. (2022, August 24). Lived experience makes the work better – by Anthony Barrows. Behavioral Scientist. https://behavioralscientist.org/lived-experience-makes-the-work-better/
- McClary, A. (2022, November 15). How does research benefit from a lived experience perspective? –. The McPin Foundation. https://mcpin.org/how-does-research-benefit-from-a-lived-experience-perspective/
- Naus, W. (2024, January 31). Because it helps us incorporate lived experience into Health Research. Why Social Science?https://www.whysocialscience.com/blog/2024/1/31/because-it-helps-us-incorporate-lived-experience-into-health-research
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.
Great reading your perspective Kiani.