AEA365 | A Tip-a-Day by and for Evaluators

TAG | human rights

Hello, I am Donna Podems, founder and director of OtherWISE: Research and Evaluation, a small monitoring and evaluation firm in Cape Town, South Africa.  We work with a wide range of international and local donors who fund a wide variety of technical interventions in areas such as environment, education, health, community development and human rights.

We encourage evaluation use through choosing and mixing different evaluation approaches that will bring credible and useful evaluation findings. Feminist evaluation is one of the approaches that I often draw upon, and this often surprises many of my colleagues.

Feminist evaluation can be useful– even for non-feminist evaluators.

Hot Tip:

  • You do not need to be a feminist to use feminist evaluation. It is important to understand that not all feminist evaluators (or evaluation theorists) agree with me. Over 18 years of conducting evaluation in more than 25 countries, I have had the privilege of working with many talented evaluators, most of whom were not feminists. In more than 15 different evaluations in Africa and Asia, my team members agreed to incorporate various elements of a feminist approach that resulted in useful evaluation processes and findings.

Lessons learned: Three lessons I have learned about addressing the question I hear the most, “How do you apply feminist evaluation if you are not a feminist?

  • Be knowledgeable about what feminist evaluation is, and is not. Many people I work with have a strong reaction to feminist evaluation and yet few can explain what the approach entails. Demonstrate how elements of the approach could enable a credible and useful evaluation.
  • Remove the label. Having two words that often elicit strong reactions together in one phrase is a challenge. Remove the label and explain the approach.
  • Adapt as needed. In my experience, feminist evaluation often provides a useful complement to other evaluation approaches.

Rad Resource:

The American Evaluation Association is celebrating the Mixed Methods Evaluation and Feminist Issues TIGs (FIE/MME) Week. The contributions all week come from FIE/MME members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · · ·

We are Silvia Salinas-Mulder, Bolivian anthropologist, feminist activist and independent consultant, and Fabiola Amariles, Colombian economist, founder and director of Learning for Impact. We have worked for several years as external evaluators for development programs in Latin America. The following ideas may help to operationalize the principles of gender- and human rights (HR)-responsive evaluation in complex, multicultural contexts.

Lesson learned: Terms of Reference (TOR) for an evaluation are not engraved in stone.

Tip: Reframe the often conventional evaluation questions and other aspects of the evaluation process to ensure that gender and HR issues surface, and evidence of change (or no change) in women’s lives is gathered. Take into account context-specific issues and gender dynamics, as well as relevant cultural patterns, such as the effects of migration in the family roles and decision-making processes within some agricultural community settings.

Lesson learned: Some stakeholders are tired of being interviewed, while others – especially rural women- are eager to be heard.

Tip: Be creative; evaluation techniques are the means not the end, and can thus permanently be created, recreated and adapted to each situation and context. For example, use “conversatorios” (round table discussions), as opposed to focus groups, to gather people with diverse backgrounds and perspectives to discuss over a particular issue of the evaluation; participants usually appreciate these reflective spaces and feel motivated to speak “outside the box”, while evaluators take a holistic overview of the topic.  Drawings, role plays and other popular education techniques may also facilitate participation of marginalized groups, including illiterate women.

Lesson learned: Answers to your questions may not contain key gender and HR issues to understand how change is occurring.

Tip: Awareness of specific cultural and gender communication patterns is crucial for an effective exchange. In any case, interviews should be dealt with as dialogues where people have the opportunity to express their priorities and points of view. Do not limit your interactions to a question-answer dynamic. Let people speak freely and “listen actively” to discover the essential. Respect and interpret the silences and do not insist on answers to your questions, rather focus on trying to understand the underlying meaning of each reaction. This will allow an eventual reconstruction of how change is occurring (Theory of Change) for the specific intervention and context, even if it has not been explicitly stated in the project design. Also, as evaluators we tend to focus on verbal communication, ignoring the importance of tone and gestures. Make sure you are alert to less explicit key messages.

Rad Resources:

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · · ·

Hello! We’re Cary Johnson, Executive Director of the International Gay and Lesbian Human Rights Commission (IGLHRC) and Efrain Gutierrez, Associate at FSG, a nonprofit consulting firm, and Co-leader of the LGBT TIG. We want to share with you some initial lessons learned and resources we have found since IGLHRC engaged with FSG in the development of a Monitoring and Evaluation System.

Lesson Learned – Funders in the field of Lesbian, Gay, Bisexual, and Transgender (LGBT) human rights advocacy are becoming more strategic and starting to request more rigorous evaluation from their grantees. FSG conducted interviews with program officers from some of the most influential foundations in the field. Most are going through strategy development processes for their human rights advocacy areas. They are also advancing their evaluation approach, which is having an impact on the level of sophistication of evaluation data requested from grantees.

Rad Resource – Evaluating advocacy efforts is challenging but the field of evaluation is developing robust resources and knowledge to help organizations achieve advocacy’s desired outcomes. IGLHRC’s efforts to promote human rights for LGBT people around the world unfold in a nonlinear fashion making it challenging to evaluate and communicate impact. Thankfully, the field of evaluation has developed tools and practices that have the ability to adapt to IGLHRC’s changing circumstances. A great resource on Advocacy Evaluation is Ehren Reed’s AEA 365 post on the topic.

Rad Resource – Explaining “attribution vs. contribution” to staff from the beginning of engagement can reduce anxiety about evaluation and keep staff engaged throughout the evaluation. As evaluators, we understand that there are many actors working together for LGBT rights and it’s difficult to establish attribution. During our first workshop, FSG discovered that IGLHRC’s staff responded very well to the concept of “attribution vs. contribution” in advocacy evaluation. FSG explained how the evaluation of advocacy efforts is about capturing the journey rather than just a specific policy change. If you want to learn more about this topic, check out the Advocacy Evaluation Wikipedia page.

Lessons Learned – If you are working with local partners in an international context make sure you engage frontline staff working on the ground. They know the cultural sensitivities that might affect evaluation and data collection practices. While developing IGLHRC’s logic model we learned that in certain regions the concept of evaluation can be perceived as a Western construct and might be seen as a way to control local advocacy efforts. This is one example of the importance of cultural competency in evaluation. As we develop the M&E system for IGLHRC, we give staff from different regions flexibility to adapt data collection activities to the regional context.

We’re celebrating LGBT Evaluation week with our colleagues in AEA’s LGBT Topical Interest Group. Follow @aeaweb on twitter this week, or subscribe to the week’s Headlines and Resources list for more LGBT Evaluation items of note. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · ·

Archives

To top