AEA365 | A Tip-a-Day by and for Evaluators

CAT | Feminist Issues in Evaluation

Hi, we are Donna M. Mertens, Professor Emeritus at Gallaudet University and a long time member and past President of the American Evaluation Association and Julie Newton, Senior Advisor in the Gender Team at the Royal Tropical Institute (KIT) in the Netherlands. We connected because of Julie’s interest in innovative evaluation strategies to measure the empowerment of women and girls, and my involvement with the development and application of the transformative paradigm in evaluation as a guide to increase the contribution of evaluation to social justice and improvement of the lives of members of marginalized communities, including women and girls. We share perspectices on the use of feminist- and gender-focused evaluation resources. Here we share our learnings and associated resources with you.

Hot Tip: CARE provides a glimpse into how to develop gender indicators inspired by an outcome mapping approach that can be found here. CARE adapted this participatory approach framed by social justice principles and inclusion of the concept of complexity. CARE demonstrates how M&E systems can be designed to enhance learning about complex processes such as empowerment and support for more flexible and adaptive programming.

Rad Resources:

The Journal of Mixed Methods Research published a special issue on research and evaluation with members of marginalized communities, including examples of the application of transformative approaches for women and girls.

We have found three books that are great resources about the use of a feminist lens in evaluation: Feminist evaluation and research, Feminist research practice, and Program evaluation theory and practice.

A new publication, Qualitative Research for Development: A Guide for Practitioners, developed for Save the Children, provides guidance to practitioners on how to integrate principles of qualitative research into monitoring and evaluation. It provides guidance on how to use participatory approaches to engage project participants (particularly children) in shaping the learning objectives of evaluations and at different stages of the project cycle.

Lessons Learned

  • The importance of being aware and reflexive on how your philosophical paradigm background will frame your whole approach to evaluation.
  • In the context of ‘empowerment’ measurement and evaluation, a transformative approach adds particular value because of its take on whose knowledge counts. A transformative approach places emphasis on how measurement (i.e. in context of evaluation, research, monitoring) can increase social justice by tackling unequal power structures that marginalize women and girls, across other intersectional markers. This involves attention to many of the issues discussed under the transformative philosophical assumptions of axiology, ontology, epistemology and methodology (i.e. dealing with ethics, whose knowledge counts, importance of context). It recognizes the value of mixed methods approaches to understanding complex issues such as empowerment.

The American Evaluation Association is celebrating Feminist Issues in Evaluation (FIE) TIG Week with our colleagues in the FIE Topical Interest Group. The contributions all this week to aea365 come from our FIE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

We are Francesca D’Emidio, Liisa Kytola, and Sarah Henon from ActionAid, and Eva Otero from Leitmotiv consultants. ActionAid is a global federation working to achieve social justice and poverty eradication by transforming unequal gendered power relations. We’d like to share an evaluation methodology tested in Cambodia, Rwanda, and Guatemala to measure shifts in power in favour of women.

Our aim was to empower women from very disadvantaged backgrounds to collect and analyse data to improve their situation. This is in line with our understanding of feminist evaluation, where women are active agents of change. Our evaluation sought to understand any changes in gendered power relations, how these changes happened, and our contribution.

To start, we developed an analytical framework based on four dimensions of power, inspired by the Gender at Work framework and the Power Cube.

demidio-kytola-henon

We then trained women leaders of collectives to use participatory tools and facilitate discussions. Leaders then identified factors that describe people with power. After this, leaders facilitated discussions with collective members, drawing community maps  to identify important spaces (home, market etc.). Using seeds women scored spaces where they currently had most power and then repeated the exercise for the past to understand what had changed. Women then told stories  in groups to explore how they gained power in these spaces.  We mapped the changes experienced by women against the dimensions of power and analysed findings with leaders. Timelines were used to understand our contribution. Finally, we triangulated the information by interviewing other stakeholders.

Lessons learned:

  • Our methodology makes power analysis simple, concrete, and rooted in contextual realities, enabling women who are illiterate to lead and participate in the process. Women quickly grasped the concepts and confidently facilitated conversations, finding the process empowering.
  • Women need to define power in their own context. We asked women to name the most powerful people in their communities to identify “factors of power.” This allowed us to understand how participants viewed power rather than imposing our own frameworks on them.
  • Designing a fully participatory evaluation process can be challenging. A shortcoming was that women did not design the evaluation questions with us. Women found analysis tiring after a long data collection process.  We need to better balance women’s active participation with their other responsibilities and logistical challenges.

Hot tips:

  • Use role play to bring abstract concepts to life. We asked groups to organise plays to represent different dimensions of power.
  • Let local people own the space. The more freedom they have, the more they are likely to get to the root of the issue by talking to each other, rather than to ActionAid. We overcame the challenge of documentation by hiring local women as note takers.

The American Evaluation Association is celebrating Feminist Issues in Evaluation (FIE) TIG Week with our colleagues in the FIE Topical Interest Group. The contributions all this week to aea365 come from our FIE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Hi! We are PeiYao Chen, Kelly Gannon, and Lucy McDonald-Stewart at Global Fund for Women, a public foundation that raises money and attention to advance women’s rights and gender equality.

Strengthening women’s rights movements is a key component of Global Fund for Women’s Theory of Change. We are charged with figuring out how to meaningfully measure progress of movement building. After two years of research, we developed the Movement Capacity Assessment Tool – an online tool that allows movement actors to assess the strengths, needs, and priorities of their movement and to use the results to develop action plans to strengthen movement capacity.

Currently in the pilot stage, the Movement Capacity Assessment Tool includes a series of questions that capture respondents’ perceptions of their movements along seven dimensions. It also captures the movement’s stages of development, because movements are always evolving.

Hot Tips

Before launching an assessment, first define the social movement you want to focus on.

  • Providing a clear definition and scope ensures that you invite the right people to participate and that the respondents are talking about the same social movement.
  • Lack of scope or difficulty in defining the movement can signify pre-nascent movements or no movement at all.

Diverse voices matter.

  • Because social movements are composed of diverse actors, our sampling approach focuses on including individuals and organizations representing different perspectives, and playing different roles within the movement. These include leaders at the center of the movement as well as those at the margins.
  • Diverse perspectives should also consider generational differences. This might be a difference between the older and younger generations of activists, between more established and newer organizations, or between old-timers and newcomers in the movement.

The tool can be used to inform planning and to measure progress over time.

Results of the assessment can help movement actors and their supporters develop a shared understanding of where the movement is, what the capacity needs are, and develop action plans accordingly.

Participants noted that the process provided space for them to reflect on their role in building movements. Some were motivated to re-engage with other movement actors to develop strategies to collectively address challenges.

Lesson Learned: The tool has its limitations – it cannot capture how different movements intersect, overlap, or exclude one another or provide a comprehensive landscape analysis of a social movement, though it may help identify new or unknown actors.

After we complete the pilot project, we plan to make the tool available to the public. If you are interested in learning more about the tool or helping us test it, please email Kelly at kgannon@globalfundforwomen.org

The American Evaluation Association is celebrating Feminist Issues in Evaluation (FIE) TIG Week with our colleagues in the FIE Topical Interest Group. The contributions all this week to aea365 come from our FIE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

We are Julie Poncelet and Catherine Borgman-Arboleda of Action Evaluation Collaborative, a group of consultants who use evaluation and collective learning strategies to strengthen social change work. Drawing from recent work with a nonprofit consortium of international NGOs engaging with women and girls in vulnerable, underserved communities in the U.S., Africa, India, and the Caribbean, we wanted to share lessons learned and rad resources that have helped us along the way.

We structured a developmental evaluation using the Action Learning Process, which focuses on on-the-ground learning, sense-making, decision-making, and action driven by a systemic analysis of conditions. We implemented a range of highly participatory tools, informed by feminist principles, to engage stakeholders in a deeper, more meaningful way. Specifically, we sought to catalyze learning and collective decision-making amongst various actors – NGOs, girls and women, and the consortium.

Lessons Learned: We have used the Action Learning Process in a number of projects, and learned valuable lessons about how this approach can be a catalyst for transformative change and development. Issues of learning versus accountability, power, ownership and participation, and building local capacity and leadership were critical this work, especially in the context of women’s empowerment, rights, and movement building. Learn more about these processes in these blog posts.

Rad Resources: The Action Learning Process draws from a number of frameworks for transformative women’s empowerment, based on research on women’s rights and women-led movements. These frameworks evidence the conditions that affect the lives of women and their communities, and that lead to scarcity and injustice.  With this in mind, we developed a series of context-sensitive tools to support women, girls, and NGOs to explore these conditions, identify root causes, and co-create ways of addressing issues affecting the lives of women, girls, and their communities. Some tools included:

  • Empathy map to provide deeper insights into the current lives and aspirations of women and girls. The insights from all the empathy maps were harvested to develop an overall framework, which were then aligned with the frameworks mentioned above.
  • Learning review guide to bring together different perspectives – staff, women, and other community actors –  to make sense of the information collected via the participatory tools, to reflect, to learn and to generate new knowledge to inform collective decision-making and ongoing planning.

The Action Learning Process attempted to redistribute the power of knowledge production from us, the evaluators, to the girls and women themselves. This was especially critical given the context: grounding the work in an analysis of women’s rights and movement building, and specifically on concepts of power and how it intersects economically, socially, culturally, and politically in women’s own lives.

The American Evaluation Association is celebrating Feminist Issues in Evaluation (FIE) TIG Week with our colleagues in the FIE Topical Interest Group. The contributions all this week to aea365 come from our FIE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hi there, I’m Heather Krause from Datassist. I’ve been working with women and data internationally for over a decade.  I’m delighted to have the opportunity to add my voice and share a bit about the importance of including quantitative data — in addition to qualitative data — in research to incorporate a feminist perspective.

Using quantitative data in nuanced and complex ways can help us better identify and understand trends in the lived experiences of women, as well as key causes of those patterns.

Lesson Learned: Complex Models = Clear Communication

Conducting evaluation from a feminist perspective calls for both conceptual understanding of feminist principles and advanced understanding of statistical methods.

While adding complexity to gain clarity may sound counterintuitive, in reality, complex statistical analysis is much better for evaluation from a feminist perspective than simple data tables. Simple tables often hide or oversimplify complex trends occurring in the real world, where a more nuanced approach can highlight gender- and other socially-based trends.

In How Not to Visualize Like a Racist, I examined the importance of a complex multivariate model in graphs that show all dimensions, rather than focusing on only one or two. Where simple tables may appear to make the data more accessible, they hide the real trends, whereas more nuanced analysis highlights and clarifies true patterns.

Bar graph demonstrating rates of poverty among different racial and gender groups

By building this statistical model that addresses the interactions between gender and other social factors, we uncover the truth.

Lesson Learned: Changing the Model Can Change the Perspective

In a world where so much data historically overlooks the feminist perspective, how can we make changes? In some cases, simple steps like changing a variable — i.e., using female respondents as your baseline, rather than male — can have an immediate and obvious effect on analysis.

Similarly, the use of moderating variables in your models provides insight into how different aspects of respondents’ social identities interact to inform their experiences. This is known as intersectionality. It is important to consider gender when analyzing data, of course; but it is equally important to understand how the interplay between gender and race, class, ability or ethnicity can affect your data.

Rad Resources:

Last October, the United Nations and Statistic Finland hosted the 6th Global Forum on Gender Statistics with the goal of bringing together policymakers and researchers to identify gender gaps and new ways of measuring and collecting data that represents both men and women equally — and ultimately, to measure global progress towards gender equality.

They made many of the presentations from that event available to the public online, providing insights from global leaders on international initiatives on gender statistics.

For more details on quantitative feminist research, I recommend Feminist Measures in Survey Research by Catherine E. Harnois.

The American Evaluation Association is celebrating Feminist Issues in Evaluation (FIE) TIG Week with our colleagues in the FIE Topical Interest Group. The contributions all this week to aea365 come from our FIE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

In this infinite mysterious magical interconnected multi universe, my existence is on planet earth as Mira Savara.  I hold a PhD, was previously with BBC World Service Trust, and am now an independent freelance consultant with the non-profit social research centre Shakti.

No… Don’t stop reading, I am making a point.

The introduction tells you that I see myself as part of my environment.

An opposing view is that I am the centre, an individual atomised self.  I control my own life.

This as an example of how life is literally perceived differently.  Different women, especially women from a different culture, have a different conception of a good life.

Research evaluation has tended to see the individual as the centre. I argue that research and evaluation need to expand the view.  Because how one sees oneself, the view we have of life, gives meaning to different things.

Lessons Learned: I was once asked to design a massive women’s empowerment programme amongst a family based economic system. “Why?” I asked.
“The women are very oppressed by patriarchy, men, and they don’t want to change,” said the tired bureaucrat.
Interesting. He knows the politically correct word, the big P, I thought. “And why are you interested?”
“You see,” he tells me, “this international organisation has given us millions of dollars for expansion. But they say that it has to go through women-only cooperatives. And the women don’t want to join. The organisation says we have to empower the women so that they see that they are oppressed, and agree for only women cooperatives.”

Sounds familiar? So what’s wrong?

Should I be telling other women that I know better than them what is better for them?
When we do feminist evaluation we need to question how the project has been conceptualised and whether there is not an imposition of a view of the world. Would that create and maintain unequal relationships between civilizations, favouring the more powerful civilization?

Feminist imperialism?

Cool Trick: Do not presume that the viewpoint you have of the world is one that everyone yearns for. Look at the world you represent from the others eyes.

Two images of women in different work contexts accompanied by two blue text bubbles with opinion on oppression

It’s a tricky tightrope, difficult situation… I hope that raising this will lead to a practical oriented debate through the work we do to develop a more holistic non imperialist perspective.

Rad Resources

I found the argument made by Saba Mahmood in “Politics of Piety: The Islamic Revival and the Feminist Subject” interesting. It shows the perspectives of women who are not seen as  “feminist” in the way usually defined.

Experiments discussed by Nisbet in Geography of Thought… How Asians and Westerners Think Differently and Why? explore how experimentally the views and the very world differs.

The American Evaluation Association is celebrating Feminist Issues in Evaluation (FIE) TIG Week with our colleagues in the FIE Topical Interest Group. The contributions all this week to aea365 come from our FIE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Hello AEA365 readers!  I’m Kate S. McCleary, Ph.D., researcher and evaluator at The LEAD Center  within the Wisconsin Center for Education Research at the University of Wisconsin-Madison.  On March 11th, my colleagues and I gathered for an office retreat to discuss our work as a Center and also to share themes in evaluation that are important to us. I shared feminist epistemologies in relation to feminist evaluation. When I began to unpack central ideas from the literature around feminist epistemologies, based on my own positionality in this world, I came up with five central themes.

Lessons Learned: For me, feminist epistemologies focus on…

  1. Women’s lives and the oppression of women and other marginalized groups: Feminist epistemologies explore difference and seek to know and understand the lived experience of those whose voices/experiences have been marginalized. Chandra T. Mohanty’s book Feminism Without Borders explores the plurality of contemporary, global feminism.
  2. Power, authority, and hierarchy: Feminist epistemologies seek to “decenter the center.” This is the title of Uma Narayan and Susan Harding’s book that explores the way feminism is enacted across borders, and in multicultural and postcolonial contexts.
  3. Relationships: The relationship that individuals have within their homes, communities, broader society, and the world hold meaning. Carol Hanisch’s (1969) claim that “the personal is political” holds true today.
  4. Facts and findings are all “value tinged”: Knowledge and knowing is socially situated; thus no one is ever able to get rid of one’s own values.
  5. Understanding the lived, quotidian experiences of women and other individuals: In 1987, Dorothy Smith wrote a book The Everyday World As Problematic that called on researchers to be attentive to the full spectrum of what constitutes women’s, and other groups, lives.

Rad Resource: There are 16 different TED Talks categorized under the topic of feminism. Feminism can be explored through media, popular culture, and literature. Watch Roxanne Gay’s talk if you question whether you are a feminist!

Hot Tip: Organize a retreat or coffee break to discuss feminist evaluation with colleagues. When we take time to learn from each other as colleagues, there is the possibility for ongoing conversation and growth.

Rad Resources: Gloria Anzaldúa’s book Borderlands: La Frontera and bell hook’s Feminist Theory: From Margin to Center are seminal pieces, and were instrumental in my early exploration of feminism.  The Handbook of Feminist Research: Theory and Practice edited by S.N. Hesse-Biber (2012) is a great resource to ponder the connection between feminist theory and practice (hence the name). Andrea Doucet and Natasha Mauthner (2012) have a useful chapter titled “Knowing responsibly: Ethics, feminist epistemologies and methodologies” which is in Ethics in Qualitative Research (2nd edition).

The American Evaluation Association is celebrating Feminist Issues in Evaluation (FIE) TIG Week with our colleagues in the FIE Topical Interest Group. The contributions all this week to aea365 come from our FIE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Hi, I am Priya Alvarez, a feminist evaluator working at UN Women. During the past 12 months, I worked on an evaluation combining feminist and systems thinking approaches with Katrina Rojas/Universalia, Annalise Moser and Bob Williams. The evaluation assessed UN Women’s contribution to UN system coordination on gender equality and the empowerment of women (GEEW).

Applying the systems approach helped us examine the complexity and diversity of scenarios and stakeholders that articulate a very dynamic system within which UN Women exercises its coordination mandate. We explored the relationships among the elements in the system, paying special attention to the multiple perspectives on GEEW and system boundaries.

The feminist approach allowed us to interrogate power relations, including the reproduction of corporate identities, to capture how organizational culture enables or disables transformation.

Lessons learned: Coupling these two approaches helped us better understand how certain dimensions of the system affect the promotion of GEEW, negatively and positively.

  • Incentives matter for horizontal coordination on GEEW among entities with vertical accountability (reporting to a headquarters).
  • A shared common vision and passion for equality creates a sense of purpose and community that enables innovation and fuels meaningful actions.
  • Expertise-based credibility is essential in the absence of financial resources as an incentive.
  • Gender champions do make a difference in mobilizing others to transform gender power relations.

Lessons learned: Ideological boundaries are important drivers for system coordination in addition to traditional boundaries such as money, resources or capacity.

  • Internal organizational integrity and consistency on GEEW are as important as the technical knowledge and approaches used to mainstream gender in policies and programs.
  • Institutional relations based on trust-based alliances, are key for countering dominant discourses and formal structures.

Rad Resources:

Srilatha Batliwala’s Feminist Leadership for Social Transformation unpacks the leadership traits and underlying principles of organizations which embody feminist approaches: less hierarchical and top down; multilayered and collective leadership, introspective and critical of their own leadership; innovative in their practices and consistently infuse a dimension of advocacy in their systems and actions.

Joanne Sandler and Aruna Rao’s Strategies of Feminist Bureaucrats: United Nations Experiences; Rosalind Eyben and Laura Turquet’s Feminist in Development Organizations: Change from the Margins and Lucy Ferguson’s This is Our Gender Person highlight the dilemmas and contradictions of institutional feminism and daily politics. They underscore the creative possibilities of networking and working from the margins of the mainstream to realize the transformative aims and power of feminism.

Bob Williams and Sjon van ‘t Hof’s Wicked Solutions: A Systems Approach to Complex Problems and Dick Morris’ Thinking about systems for sustainable lifestyles helped us understand systems as subjective and changeable and balanced our approach to complexity.

The American Evaluation Association is celebrating Feminist Issues in Evaluation (FIE) TIG Week with our colleagues in the FIE Topical Interest Group. The contributions all this week to aea365 come from our FIE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

I’m Rita Fierro from Fierro Consulting and I specialize in working with vulnerable populations in a more inclusive way. An inclusive conversation is one in which differences in perspectives are leveraged as opportunities instead of threats or ignored.

Lessons Learned: To suggest to a client that a gender-responsive evaluation or an inclusive facilitation process is needed (along gender, race, ethnic, class, or sexual orientation lines), I must know how to advocate for inclusion. The client has to see the benefits of including more stakeholders to participate in the planning stages of the work:

  • Effective strategies. If only men or only professionals are involved in planning a strategy or an initiative for a more diverse audience, the approach will be ineffective once it faces a wider audience. Learning how to talk with someone who doesn’t move, talk, think, or look like you to get back insightful feedback early on is essential to having a positive impact.
  • Sustainability. When we build our capacity to have inclusive conversations, we learn to see common threads between our passions and those of others. We build more effective partnerships and discover more support than we thought possible. Initiatives take legs of their own; people continue to invest when the funding ends.
  • Wisdom. Often people on the margins disengage because they understand and see things that people in charge missed. Without an opportunity to voice their critique, they give up. Inclusive conversations help surface hidden information. For instance, recipients may voice threats of gender violence as a result of participating in the evaluation.
  • Creativity. Building your ability to plan and talk across differences means building your capacity to be creative, flexible, original, genuine, and positive at the same time. These skills are helpful in other areas, too.
  • Authenticity. Misunderstandings among differences often come from some pushing to be heard or others pulling away for not being heard. Emotions build up and fester. When we are more inclusive, we create opportunities to voice tensions and listen deeply so people can be seen and heard. The result is often releasing of fear, anger, and shame and more compassion. The group can get much closer.

All the examples above focus on gender, but you can use other examples from your own context to advocate for inclusion. Make your own list: what do you see as the benefits of inclusion?

Rad Resource: Don’t know what an inclusive conversation looks like? Click here.  Want to build your capacity to have inclusive conversations? Subscribe to my blog. I’m writing 13 posts on inclusive conversations.

The American Evaluation Association is celebrating Feminist Issues in Evaluation (FIE) TIG Week with our colleagues in the FIE Topical Interest Group. The contributions all this week to aea365 come from our FIE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Hi there!  I’m Michael Bamberger and I have worked on gender evaluation with civil society, UN, foundations, and bilateral donor agencies.  The longer you work in this field, the more you recognize the complexity of the processes and outcomes you are trying to understand.

There is growing recognition that many development programs are complex. They have many components, multiple funders and implementing agencies, and even more stakeholders – usually with different objectives.   Programs are also influenced by multiple external factors (e.g. economic, political and institutional), affecting how they are designed and implemented, as well as their outcomes.  Finally, the processes of change result from multiple, non-linear causal chains, which are difficult to identify and measure.

All these dimensions are critical for evaluating gender equality and women’s empowerment (“gender” for short) dimensions of development programs.  The following are some of the common complexity issues in gender evaluation:

  • Societies have multiple social control mechanisms determining the “appropriate” behavior of women and men in the household, community, work place and the wider political arena. Many gender programs must include multiple components to simultaneously address a wide range of social control mechanisms.  Consequently,  innovative, complexity-responsive evaluation designs are required.
  • Given the many mechanisms of social control that resist change, a common phenomenon is “one step forward and two steps back” as initial successes in one area (such a women’s access to mobile phones or access to markets through micro-credit) can encounter “push-back”. This means that recursive, non-linear models of change must be used.
  • Gender programs also produce many subtle forms of behavioral change (such as how spouses make decisions on household expenditures), that are difficult to observe and measure.

Lesson Learned:  I’ve been working with some of my colleagues on a five-step approach to the evaluation of complex, gender-focused development programs.  It includes elements of “unpacking” and “reassembling” concepts and methods.

  • Step 1: Holistic analysis of the nature of the program and the context in which it operates.
  • Step 2:“Unpacking” programs into a set of components, each of which can be analyzed separately. This approach permits the use of conventional evaluation methods for evaluating each component
  • Step 3: Identifying the unit of analysis  for each component evaluation
  • Steps 4 and 5: “Reassembling” the findings of the individual evaluations to understand the overall effects of the program within the broader context within which it operates.

We are hoping this will assist evaluators in ensuring that gender evaluations take into consideration the complex framework within which these programs are implemented and evaluated.

The American Evaluation Association is celebrating Feminist Issues in Evaluation (FIE) TIG Week with our colleagues in the FIE Topical Interest Group. The contributions all this week to aea365 come from our FIE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Older posts >>

Archives

To top