Hi, we’re Naomi Ali and Dayna Gallagher. We work at Mathematica, a research organization that uses data and analytics to provide evidence-based insights to foundations, government agencies, and other organizations.
For six years, we’ve partnered with Boston Children’s Hospital to monitor and evaluate the contributions of community organizations that receive funding through their Collaboration for Community Health (the “Collaboration”). In this blog post, we share what we’ve learned through this partnership about the equity considerations of demographic data collection.
In recent years, we’ve seen grant makers increasingly adopting a racial equity lens to improve the equitable distribution of resources in communities. Some funders—including the Collaboration—ask organizations to report on the demographic data of their program participants. This is a useful tool for understanding who funding is reaching, but funders are challenged to balance this use against the equity and resource considerations of asking community organizations to collect demographic data.
Here are a few lessons learned in how Mathematica and the Collaboration implemented demographic data collection requirements:
Lesson Learned #1: Allowing flexibility in demographic data requirements acknowledges organizations as experts in their communities.
Some community-based organizations already collect demographic data as part of their intake or evaluation processes. Others may refrain from collecting this data based on their community or program dynamics. For example, some organizations find collecting demographic data like race/ethnicity disruptive to the relationship building process between participants and program staff, particularly for groups with historical mistrust of institutions. Other organizations describe how some participants perceive these intake questions as asking them to “justify” why they should receive services.
Funders can address these concerns by implementing flexible requirements for how organizations can report demographic data. Allowing organizations to use secondary data sources like census data or school enrollment data, for example, addresses these concerns while still gathering useful information on the population reached.
Lesson Learned #2: Funders have opportunities to reduce the burden of demographic data collection and support community use of this data.
Developing a process to collect and report demographic data can be a big lift, particularly for smaller organizations. Funders can address this from two ends: reducing the burden and improving the benefit. To reduce the burden, funders can provide opportunities for organizations to share strategies on collecting demographic data, develop technical resources to support implementation (such as the Collaboration’s Demographic Data Special Issue Brief), and adapt reporting requirements in response to feedback. To improve the benefits of collecting this data, funders can support organizations in using demographic data for other purposes, like improving programs and advocating for resources.
Lesson Learned #3: Understanding that demographic data has limitations and may not be representative of community contexts.
Depending on a given program’s model and context, demographic data on participants may not reflect the true diversity of those reached. For organizations providing direct services, demographic data collected from participants won’t capture a program’s indirect impacts on family members and neighbors of participants. Organizations engaged in systems-level work may not be able to capture demographic information directly from those ultimately impacted by their programs and instead make use of secondary data.
Considering this, funders should reflect on whether demographic data on participants who access grant-funded services fully reflects the communities benefiting from funding. Funders should also consider whether demographic data collection is sufficient in gauging organizations’ commitment to equity. Other equity considerations include the diversity in organization leadership, implementation of community-tailored, person-centered programming, and engagement with community members to improve programming. By acknowledging these nuances, organizations and funders can engage in open dialogue to improve demographic data collection as an equitable practice.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.