My name is Heather Krause. As a data scientist for the Ontario Syrian Refugee Resettlement Secretariat, part of my job is to design ways to harness data to measure how successfully refugee resettlement is going, as well as what programs and services are working well and which ones have gaps.
Using data to advocate for vulnerable groups can be tricky. For starters, not everyone in vulnerable groups is wild about the idea of having data collected on them. Secondly, there is usually a broad range of stakeholders who would like to define success. Thirdly, finding a comparison group can be challenging.
To avoid placing additional burden on vulnerable people, one option is to use public data such as Census, school board, or public health data. This removes both the optical and practical problem of collecting data specifically from a unique or small population. Public data can often be accessed at a fine enough level to allow for detailed analysis if you form partnerships and data sharing understandings with the public data owners. An agreement to include their questions of interest in your analysis and to share your findings with these often-overburdened organizations goes a long way to facilitating data sharing agreements.
Once you have access to public data, deciding on indicators of success is the next step. For example, accessing day care and working outside the home is seen as empowerment by some women, but not others. Neither of these is a neutral measure of success. To make matters more complex, diverse stakeholders often define success differently – from finding adequate housing to receiving enough income to not receiving social assistance.
Lesson Learned: I have found that the best way to handle this is to allow the voices of the vulnerable group to guide the foundation of how success is defined in the measurement framework. Then to add a few additional indicators that align with key stakeholders’ interest.
Finally, once you have data and indicators selected you need to devise a way of benchmarking success with vulnerable groups. If, for example, the income of refugees is being measured – how will we know if that income is high enough or changing fast enough? Do we compare their income to the general population income? To other immigrant income? To the poorest community income?
Hot Tip: There is no simply answer. The best way to deal with this is to build multivariate statistical models that include as many unique sociodemographic factors as possible. This way you can test for differences both within and between many meaningful groups simultaneously. This helps you avoid false comparisons and advocate more effectively for vulnerable populations using data.
The American Evaluation Association is celebrating APC TIG Week with our colleagues in the Advocacy and Policy Change Topical Interest Group. The contributions all this week to aea365 come from our AP TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.