I’m Rasec Niembro, an evaluator with over 6 years’ experience in Latin America, currently working as a consultant at the Inter-American Development Bank’s Office of Evaluation and Oversight.
In evaluation, we know that what cannot be measured cannot be improved. As practitioners, we share the responsibility not only in terms of producing objective information but also promoting the generation of useful and reliable data. This kind of concern is even more relevant in Latin America given highly unequal socioeconomic conditions.
Lesson Learned:
Recently, I worked in the Evaluation of the IDB’s Support for Gender and Diversity and was able to identify that regionally comparable statistics on women and indigenous outcomes remain scarce in crucial areas, impeding efforts to close gender and ethnic inequalities as well as monitor and evaluate numerous indicators and results. For example, Mexico is considered by the World Bank an upper middle-income country, however, using disaggregated data it’s possible to identify massive differences among populations, this variance implies that the Human Development Index in a municipality in Mexico City equals more than double than an indigenous municipality in Oaxaca. This distance is the same between the Netherlands and South Sudan, located in 4th and 181 of the world and shows that in some areas a poverty gap prevails ten times more among indigenous women compared to non-indigenous men. Without this information it would be impossible to design, implement and evaluate an efficient program targeted to reduce poverty.
Some challenges with disaggregated data that persist in Latin America are: lack of awareness of the value or importance of sex and ethnic disaggregated data, limited collection due to systems not set up to capture data at the individual level and, probably most noteworthy is that even if some data was available, it lacked quality or was simply erroneous. Despite numerous challenges, the evaluation also generated a variety of solutions and alternative research methods.
Cool Trick:
The specific data disaggregation needs at the country level must be taken into account at project planning and design stages. Where standard sample design fails to produce sufficient representation of specific populations of interest, alternate sampling and data collection approaches should be considered. These methodologies may include oversampling (increasing the number of units within an established sample design to increase the likelihood of populations of interest being included), targeted sampling (using existing information in census data or administrative records about the geographic distribution of the population of interest) or comparative surveys of target population groups with other population groups living in the same areas.
Implications for Evaluators
Collecting disaggregated data may imply more time and costs, even other complications because it can include aspects of biology, identity, and culture, among other factors. However, evaluators can be a vital force to promote the use and generation of reliable and updated disaggregated data. We must remember that using disaggregated data to evaluate interventions is essential to identify more accurate results and to make visible vulnerable populations because if they are seen and understood, they are more likely to be located at the center of policymaking and therefore evaluation.
The American Evaluation Association is celebrating Latina/o Responsive Evaluation Discourse TIG Week. The contributions all this week to aea365 come from LA RED Topical Interest Group members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.