Hello again, we are Emily Gates (assistant professor at Boston College), Andrés Castro Samayoa (associate professor at Boston College), and Min Ma (Founder, MXM Research Group). We break from our roles as SCENE coordinators to share a few thoughts about the relationship between funding, equity, and evaluation.
When we first asked evaluators about this relationship, we heard again and again about how funders are the primary barrier to equity in manifold ways. This has begun to change thanks to multi-pronged conversations led by a variety of actors, from the Equitable Evaluation Initiative to the U.S. Office of Management and Budget’s charge evaluating the equitable provision of services to underserved communities.
Our thoughts are based on a study Emily and colleagues conducted in 2020 in New England (full report here) and highlights from two events held by our collaborative, Co-creating Contractual & Organizational Support (watch here) and an AEA2022 panel. These events facilitated dialogue between funders and evaluators working at large firms and small or independent consultancies.
Lessons Learned:
Some of the ways funders can be barriers to equity:
- Setting the purpose and questions for the evaluation unilaterally. Telling evaluators what to investigate and how (using narrow designs or methods) undermines evaluators’ professional autonomy. It can further inequity by excluding intended beneficiaries from shaping the questions and failing to recognize its systemic drivers. Evaluators in positions to push back and raise alternatives tend to be from large firms with power and those in financially secure positions, which can marginalize small and independent consultancies.
- Making the timeframe too short and the budget too small. Asking evaluators for quality work in unrealistic time frames and budgets encourages evaluators to cut corners, which might mean reducing participation in the evaluation process or failing to share data back to communities. It also asks evaluators to do equity work on the side or outside of the contract, which can contribute to stress and burnout.
- Expecting evaluators of color to represent groups or lead equity work. Several interviewees of color shared examples of racial ignorance and discrimination, including being treated as a token representative and being expected to do more as evaluators of color, which created inequities for them as evaluators.
Hot Tips:
Workarounds and ways to reshape funder-evaluator relationships:
- Evaluators leverage power. Ideas include prioritizing Request for Proposals (RFPs) focused on inclusion, diversity, and equity or that align with your values; leaning to leadership within your team or organization (or being those leaders) who speak up for equitable work; having up-front conversations between funders and evaluators at the beginning and throughout the work; and ensuring adequate budget before taking on a contract.
- Prioritize diversity and critical reflection within evaluation teams and organizations. Evaluators emphasized the importance of having members with diverse cultural and linguistic expertise, inviting in community and new evaluator voices, and fostering critical, self-reflective, and learning cultures. Organizational helpful factors include having missions and values aligned with equity, leadership support, internal diversity, equity, and inclusion work, and dedicated staff time.
- Educating and engaging funders. Evaluators suggested a need to educate funders and engage new financial partners.
Ideas for further conversation and action
- Interrogate inequity within the evaluation marketplace. A handful of large firms get most federal funding, as they have infrastructure that sets them up with advantages. How these firms contract with small firms and independent contractors can perpetuate or interrupt inequity. New funding streams from corporations with social impact agendas raise new questions for equitable evaluation.
- Set new professional standards. Interviewees pointed to ways the evaluation field itself can constrain equity in its professional culture and norms that privilege scientific knowledge and objectivity, assumed evaluator roles as experts, and a sense of isolation, competition, and little sharing between evaluators.
Rad Resources:
- We All Count’s Introduction to the Funding Web
- From Contractors to Conduits: An Exploratory Dialogue among Funders and Evaluators
- Radical Reimagining podcast episode with Rory Neuner
- Intention 2 Impact’s AEA 365 Post on Developing More Intentional Budgets
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.