We are Rebecca Stewart, Chief Practice Officer, and Kate Noble, Research Analyst, both with The Improve Group, a firm based in St. Paul, Minnesota. The International Year of Evaluation’s focus on equity prompted us to look at our practice with fresh eyes.
First, we examined how our existing practices help to highlight issues of inequity:
- Evaluating programs that target underserved communities
- Learning from those not well served by systems
- Looking at differences in program impacts by demographic information, such as race and gender
- Valuing knowledge from people who experience the issues being studied in addition to those who have a professional experience with the issue
- Working to understand stakeholders well enough to design inclusive evaluations
- Highlighting “interocular results” (e.g., important equity-related issues discussed by only a few participants)
Lessons learned: We found it useful to ask ourselves “When should we be pushed beyond inclusive practices to focus evaluation questions on equity?” We determined issues of equity should be considered in these all-too-common circumstances:
- When the community you are studying has a history of marginalization in the context of the study.
- When working on issues for which related outcomes are inequitable for groups with a shared characteristic(s).
Lessons learned: Reflecting on when to address issues of equity led us to recognize the need for additional tools. For example, we are creating talking points to help us advocate with clients or potential clients to include evaluation questions about equity– even when that is not what they anticipated. Most of our clients are supportive, but when it may require extra time and money, we need to make a compelling case.
Hot Tip: Set aside time to reflect. We created a small working group of particularly interested folks to read related articles and discuss when and how to focus our attention towards equity, then we engaged the broader staff in honing ideas.
Hot Tip: To us, it seemed like we could infuse an equity focus into existing processes and methods by adding critical questions at certain junctures, especially in project design. For instance, when designing a recent project, we suggested adding an evaluation question around the degree to which unintended bias or discrimination was a barrier to accessing services.
Rad Resource: CREA’s 2016 conference on equity.
Rad Resource: Public Policy Associates (PPA) and ¡Milwaukee Evaluation! recently held a webinar series on equity and evaluation. PPA is also holding a workshop on using a “racial equity lens” at AEA.
Rad Resource: UNICEF’s guide to designing and managing equity-focused evaluations, and their compilation of essays on the topic.
Rad resource: My M&E defines equity-focused evaluations here.
We’re celebrating 2-for-1 Week here at aea365. With tremendous interest in the blog lately, we’ve had many authors eager to share their evaluation wisdom, so for one special week, readers will be treated to two blog posts per day! Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.