Hi! This is Michael Bamberger, an Independent Consultant specializing in the evaluation of social development and gender programs. Over the past few years I have worked with United Nations organizations, bi-lateral programs and NGOs helping strengthen their evaluations of their gender equity policies.
Most international development agencies have now defined the promotion of gender equality (or gender equity) as one of their development goals, and most conduct periodic evaluations to assess the extent to which their programs and policies contribute to strengthening the economic, social and political empowerment of women and the reduction of the differences between women and men on these dimensions. However, many of these evaluations tend to over-estimate the positive effects of their programs on promoting gender equality and frequently under-estimate or even ignore some of the negative outcomes of these programs. Frequently the evaluations only interview the women participating in the project, and produce glowing reports on the significant benefits, but fail to talk to women who did not participate. The failure to identify negative outcomes is unfortunately very common and has serious implications.
Hot Tips:
- Build into the Terms of Reference for the evaluation a requirement that the evaluators, even when working on a limited budget and under time-constraints, must identify and interview women (and where appropriate men) who did not benefit from the project.
- Assess carefully the evaluation methodology to ensure that it is capable of identifying unintended outcomes. Many evaluation methodologies such as results-based evaluations, many theories of change, and most experimental and quasi-experimental designs only measure the extent to which intended results have been achieved and are not able to capture unintended outcomes.
- Be aware that many evaluations only obtain information from people directly involved in the program, most of whom will be reluctant to criticize the program which pays their salary.
- In most communities there are people who are familiar with the program being evaluated and its effects, but who are not directly involved and who are able to provide a more balanced perspective. Examples include: the district nurse (who usually knows almost all of the women in the community), the police chief (a useful source in cases of domestic violence), women’s organizations, school teachers and local religious leaders.
- Use a mixed methods design that combines quantitative and qualitative data and that emphasizes the importance of triangulation to increase validity by systematically comparing information collected from different sources.
Rad Resources: The World Bank “Gender and Transport” website illustrates many of the challenges and unintended consequences for women and girls of transport initiates which are assumed to be “gender neutral”. Module 2 identifies the challenges and Module 5 identifies tools, including research tools, for promoting gender equity in this sector.
The American Evaluation Association is celebrating Feminist Issues in Evaluation (FIE) TIG Week with our colleagues in the FIE Topical Interest Group. The contributions all this week to aea365 come from our FIE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.