Using Tracer Methodology to Understand the Lived Experience of Diverse Individuals by Michael Valenti and April Wall-Parker

Hello, AEA365 community! Liz DiLuzio here, Lead Curator of the blog. This week is Individuals Week, which means we take a break from our themed weeks and spotlight the Hot Tips, Cool Tricks, Rad Resources and Lessons Learned from any evaluator interested in sharing. Would you like to contribute to future individuals weeks? Email me at AEA365@eval.org with an idea or a draft and we will make it happen.

---

Hello! We are Michael Valenti, PhD, and April Wall-Parker, MS, from Pressley Ridge, a nonprofit social services agency. Our organization is committed to providing equitable services for all and has made it a strategic priority to ensure that our programs are safe and nurturing spaces for every individual. As evaluators, we analyze performance data for our program leaders to ensure that we are having the right impact on our communities. A few years ago, we began disaggregating our performance indicators by race and gender so our leadership teams can see how successful our programs are for diverse people.

Disaggregating data was a great first step towards evolving our evaluation practices to be more mindful of the disparate outcomes often found in the child and family services sphere. Disaggregating data helps us to identify the “who” question; that is, for whom are our programs working? It helps us uncover unequal outcomes, but it doesn’t give us the information we need to effect change in those outcomes. We need answers to the “what,” “when,” “where,” “why,” and “how” questions to uncover the root causes. Enter the equity tracer.

The equity tracer is a tool we developed to help track an individual’s experience with discrimination during their interactions with different systems of care. It all started with a literature review focusing on common sources of bias in human-facing systems (e.g., healthcare, child welfare, juvenile justice, etc.). The tracer process begins with a thorough record review of all relevant client documentation, focusing on the seven areas of bias that appeared most often in our review. One area of bias is the type of language used, as some practitioners may use harmful or stereotypical language when documenting the behavior of diverse individuals (one psychiatric evaluation we reviewed referred to a person’s behavior as “gang-like;” this person was not involved with gangs in any way).

After the record review, we meet closely with the individual’s treatment team to discuss our findings and add context that might be missing from the review. We document these findings on a scoring sheet which summarizes events that may be discriminatory in nature. We end the process by facilitating a discussion with program staff about how these various sources of bias might affect those we serve, strategies for enacting change, and how we can advocate for our clients.

Hot Tips

• To date, every tracer we’ve completed uncovered potential evidence of discrimination, so be prepared to have some difficult, but courageous conversations. It helps if your staff have DEI training and can view experiences from an “equity lens.”
• Scores don’t really matter. We don’t want our staff to get hung up on the numbers (as evaluators, you can imagine the existential crisis this prompted). The tracer process uncovers events that we never want anyone to experience. So how much discrimination isn’t our main concern. All that really matters is an individual’s experience and how it has affected them.
• This isn’t rocket science. You don’t need a tool to do this work yourselves. One thing we’ve learned is that evidence of discriminatory treatment is hiding in plain sight if you take the time to look for it.

Lessons Learned

• Most clients experienced discrimination before they started services with us, indicating that many individuals are exposed to bias at an incredibly early age.
• Our first attempts with the equity tracer did not include reaching out to clients about their own experiences with discrimination. Recently, we’ve begun to work directly with clients (duh) to ask if they feel like they’ve been treated unfairly or differently. Using more human-centered research methods is influencing how we think our research can best serve our communities in the future.
• Doing this work requires a culture that honestly promotes equity and inclusion. Sharing tracer results is powerful and leads to increased understanding. It motivates our staff to think critically about how they can help prevent inequitable treatment in related systems.

---

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.