AEA365 | A Tip-a-Day by and for Evaluators

CAT | Feminist Issues in Evaluation

Greetings from Music City! I am Kathryn Bowen, Vice President for Research & Evaluation at Centerstone Research Institute. I am involved with the planning, implementation, and reporting of multiple program evaluations. Universal professional values like honesty, integrity, accountability, tolerance, and respect for people influence all my work. My feminist values are aligned with fairness, social justice, equity, and empathy regardless of political, social, economic, geographic, gender, ethnic, and age differences.  I intentionally design evaluations to increase the likelihood that the data collected, analyzed, and reported help me understand the multiple realities and lived experiences of women and sensitizes me to social structures that perpetuate inequity, oppression, social injustice, and powerlessness of women. This is done most commonly by including multiple and mixed methods in my evaluation/analysis plans. My ultimate aim is to generate knowledge to create change that makes a difference in the lives of women. A disclosure and conscious decision on my part is that in some program contexts I need to conduct these feminist evaluations without out “framing it” as feminist.

Lessons Learned:

  • I have found that using the term feminist can undermine the intent of the evaluation work. Instead, I strive to be clear about values and objectives of the evaluation.
  • Depending upon the program context and the culture/perspective of program participants, my approach as feminist might exclude rather than include women impacted most significantly by the program.
  • It is important to recognize that my concerns as an educated, white, middle class woman may not be normative for traumatized women enrolled in co-occurring mental health and substance abuse treatment programs.
  • Systematic oppression can be totally invisible to women who have internalized it from the cradle.
  • To integrate feminist guidelines that help to frame the evaluation planning and implementation it is best to be transparent about what I mean by “feminist evaluation” rather than being strident about using the term “feminist” or labeling myself a “feminist evaluator”.

Hot Tip:

  • Feminist evaluation principles need to be reflected clearly and words marked by authenticity rather than a label.
  • As a feminist evaluator, clearly frame your values, seek to understand the values of program stakeholders, and establish ways to communicate shared and divergent values in the process. This can help you understand lived experiences and identify structural inequality that exists in organizations, institutions, governments, or social networks where embedded bias provides advantages for some members and marginalizes or produces disadvantages for others.

Rad Resources:

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Hi, I’m Katherine Hay. I’ve spent the last 15 years in India working on development, research, and evaluation.

Lessons Learned:

  • A mantra I use all the time is:  ‘there is no gender neutral policy, program, or evaluation.’  If I hear one of these things described as ‘gender neutral’ I start to probe.  Usually when an intervention is called ‘gender neutral,’ it is actually gender blind.
  • South Asia, my home and the place I work, has the worst gender inequities in the world.
  • Evaluation can reinforce or reflect social inequities – or it can challenge them. I want to play a part in challenging them. To do that, evaluation has to help us figure out what shows promise in shifting inequities and what does not.  This is what draws me to feminist evaluation.
  • Mainstream development, and by extension mainstream evaluation, grapples with mainstream questions.   This has resulted in designs, approaches, and tools which are not particularly well suited to understanding inequities.   Feminist analysis brings inequity to the foreground.

Hot Tips:

  • I’m often asked, ‘But how do you do feminist evaluation?’  There are no shortcuts.  The answer is, ‘by applying feminist principles at different stages in an evaluation.’   For example:
  1. At the start of the evaluation feminist analysis can be used to ask, ‘whose questions are these?’ and, ‘whose questions are being excluded?’
  2. A rigorous feminist evaluation uses the mix of methods that matches the questions.  But some designs factor out the perspectives of marginalized groups.  Feminist evaluation designs include them.
  3. At the judgment stage, feminist evaluations recognize that there are different and often competing definitions of success in development interventions. Feminist analysis brings these differences to the surface for debate.
  4. At the use stage, feminist analysis brings recognition that particular pathways may be strategic, blocked, or risky. A feminist approach also brings responsibility to take responsible action on findings.
  • Get Involved. Peer support has been invaluable to my evaluation practice.  I’m part of a group in South Asia trying to strengthen our work through feminist analysis.  We share our designs, instruments, processes and challenges.  We are critical but supportive.  Being part of this group reminds me why evaluation matters. Try to find a group of peers to challenge and inspire you.  If you want to share resources or get in touch, we have a Feminist Evaluation website.

Rad Resources :

The American Evaluation Association is celebrating the Mixed Methods Evaluation and Feminist Issues TIGs (FIE/MME) Week. The contributions all week come from FIE/MME members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

My name is Elizabeth (AKA: Bessa) Whitmore. Now a retired Professor from the School of Social Work, Carleton University, Ottawa, Canada, I have been a member of the Feminist TIG since its inception. The following entry draws on a chapter I am writing entitled “Researcher/evaluator roles and social justice” in a forthcoming Handbook on Feminist Evaluation (edited by Denise Seigart, Sharon Brisolara and Sumitra SenGupta).

Hot Tips:

  • There are a range of roles played by a feminist evaluator, including facilitator, educator, collaborator, technical expert/methodologist, and activist/advocate. Not everyone can do everything equally well, so self-knowledge and confidence in one’s strengths (and limitations) is essential. The personal characteristics, experience and preferences of the evaluator will dictate which role(s) she/he best plays. It is critical to recognize that what role the evaluator plays and how, is intimately tied to her/his own worldview, history, and biography. There is no objectivity; we need to be aware that we are deeply grounded in our own location and life experience.
  • Good “people skills” are essential when engaging stakeholders in the process. These include active listening, cultural sensitivity, non-verbal communication, motivating participants, coordinating relationships, encouraging interactions, supporting others’ ideas, and an ability to reflect critically on one’s own reactions and behavior.
  • Having fun: We should not dismiss the importance of fun in this work. “If I can’t dance, I don’t want to be part of your revolution” said Emma Goldman back in the 1930s. Long hours without some laughter tend to burn people out, or they just drop out.

Cool Tricks: Here are some questions one might ask when planning and implementing a feminist evaluation:

  • In what ways are women (men, bisexual and transgendered people, etc.) treated differently within the program, and how do their experiences and outcomes differ? In what ways do class, race, and gender combine to expand or contract possibilities for participants?
  • Are both women and men being consulted about objectives and activities? Which women, and which men? Has the potential for community resistance to women’s empowerment activities or organizational resistance to female managers been assessed?
  • Did the project have any unexpected (positive or negative) social and gender equity outcomes?

Lessons Learned:

  • A feminist lens enhances validity in all evaluation approaches. For example, an experimental design pays attention to the sample distribution among men and women, considers gender related factors in the questions asked, and in data analysis. A utilization focused evaluation attends to the gender (and other) distribution (in decision-making). Social justice approaches (such as empowerment, participatory, collaborative, transformative, etc.); consider the equality and quality of gender participation.
  • Get involved: A good place to discuss these and other issues is the Feminist TIG.

The American Evaluation Association is celebrating the Mixed Methods Evaluation and Feminist Issues TIGs (FIE/MME) Week. The contributions all week come from FIE/MME members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

Greetings, I am Denise Seigart, Associate Dean for Nursing at Stvenson University.  Like a great novel, a great feminist evaluation creates the conditions for learning and change, particularly for the benefit of women. In 2008-2009 I implemented a feminist evaluation to study school-based health care in the United States (U.S.), Canada, and Australia. In the process of implementing a qualitative study of school-based health care, I utilized a feminist lens and feminist methods, including reflexivity, interviews focused on active listening and the experiences of the interviewees, collaborative examination of the data with interested stakeholders and other feminists and non-feminists, and diverse dissemination of the results for the purpose of promoting dialog, health care reform, and social justice for children. It was my intent to create conditions for a critical feminist exploration of school health care for children across the three countries, to share this information, and ultimately, promote community learning, action and change.

Lessons Learned:

  • Feminist evaluation is like other evaluation. It is concerned with measuring the effectiveness of programs, judging merit or worth, and examining data to promote change. The difference between feminist approaches and other evaluation models generally lies in the increased attention paid to gender issues, the needs of women, and the promotion of transformative change.
  • Feminist evaluation is interested in promoting social justice for women, but includes other oppressed groups as well. Attention is paid not only to gender but to race, class, sexual orientation, and abilities. In my study, I interviewed 73 school nurses, parents and administrators in Canada, Australia, and the U.S. regarding the presence and quality of school health care in their countries. I paid particular attention to emerging themes that indicated problems with racism, sexism, and classism, and asked additional questions as these emerged. For example, it was apparent that certain groups had more difficulty accessing health care in schools (aborigines, children with special needs) and that depending on the school district, services could vary widely. Teachers (largely women) were often asked to act as health care providers to save school districts money, and nurse practitioners (largely women) experienced difficulty gaining access and approval to provide care in schools.
  • To implement a feminist evaluation, think carefully about the questions you want to ask, the methods you want to use, and the setting. Some facilities may bar access to evaluators who declare themselves as feminist, so the language you use should be carefully chosen. Be sure to involve other feminists and non-feminists, so when planning your design or analyzing data, you can check for misinterpretations or “what would a feminist see?”

Rad Resources:

The American Evaluation Association is celebrating the Mixed Methods Evaluation and Feminist Issues TIGs (FIE/MME) Week. The contributions all week come from FIE/MME members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Hello! I am Alessandra Galiè, a PhD Candidate at Wageningen University in the Netherlands. From 2006 to 2011 I collaborated with a Participatory Plant Breeding programme coordinated at the International Centre for Agricultural Research in the Dry Areas (ICARDA) to assess the impact of the programme on the empowerment of the newly involved women farmers in Syria. The findings helped to understand how empowerment as a process can take place, and were useful to make the programme’s strategies more gender-sensitive. I chose to work with a small number (Small-N) of respondents (12 women) and a mixture of qualitative methods to provide and in-depth understanding of changes in empowerment as perceived by the women themselves and their community.

Lessons Learned

  • Small-N research is valuable. Small-N in-depth research is often criticised for its limited external validity. However, it was an extremely valuable methodology to explore a field of research that is relatively new with the aim of providing an understanding of complex social processes, of formulating new questions and  identifying new issues for further exploration.
  • Systematic evaluation should include empowerment. Empowerment is an often cited impact of development projects but rarely the focus of systematic evaluation. Assessing changes in empowerment required an approach that was specific to the context and intervention under analysis and that was relevant to the respondents and their specific circumstances. This revealed different positionalities of women in the empowerment process and the inappropriateness of blue print solutions to the ‘empowerment of women’.
  • Measure gender-based implications. An analysis of the impact of a breeding programme on the empowerment of women showed that ‘technical interventions’ have gender-based implications for both technology effectiveness and equity of development concerns.

Resources

The American Evaluation Association is celebrating the Mixed Methods Evaluation and Feminist Issues TIGs (FIE/MME) Week. The contributions all week come from FIE/MME members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

Hello, I am Donna Podems, founder and director of OtherWISE: Research and Evaluation, a small monitoring and evaluation firm in Cape Town, South Africa.  We work with a wide range of international and local donors who fund a wide variety of technical interventions in areas such as environment, education, health, community development and human rights.

We encourage evaluation use through choosing and mixing different evaluation approaches that will bring credible and useful evaluation findings. Feminist evaluation is one of the approaches that I often draw upon, and this often surprises many of my colleagues.

Feminist evaluation can be useful– even for non-feminist evaluators.

Hot Tip:

  • You do not need to be a feminist to use feminist evaluation. It is important to understand that not all feminist evaluators (or evaluation theorists) agree with me. Over 18 years of conducting evaluation in more than 25 countries, I have had the privilege of working with many talented evaluators, most of whom were not feminists. In more than 15 different evaluations in Africa and Asia, my team members agreed to incorporate various elements of a feminist approach that resulted in useful evaluation processes and findings.

Lessons learned: Three lessons I have learned about addressing the question I hear the most, “How do you apply feminist evaluation if you are not a feminist?

  • Be knowledgeable about what feminist evaluation is, and is not. Many people I work with have a strong reaction to feminist evaluation and yet few can explain what the approach entails. Demonstrate how elements of the approach could enable a credible and useful evaluation.
  • Remove the label. Having two words that often elicit strong reactions together in one phrase is a challenge. Remove the label and explain the approach.
  • Adapt as needed. In my experience, feminist evaluation often provides a useful complement to other evaluation approaches.

Rad Resource:

The American Evaluation Association is celebrating the Mixed Methods Evaluation and Feminist Issues TIGs (FIE/MME) Week. The contributions all week come from FIE/MME members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · · ·

Hello!  We are Donna Mertens and Mika Yamashita, Chair and Program Chair of Mixed Methods Evaluation TIG.  This week, we offer five posts written by Feminist Issues in Evaluation TIG members. Why is the Mixed Methods TIG co-hosting this week with the Feminist Issues in Evaluation TIG?  Because this week’s posts touch upon issues associated with theoretical and paradigmatic choices and their implications to evaluation design and methods.   “Mixed methods” may give you an impression that it is all about techniques of using quantitative and qualitative methods in one evaluation study.  It is one area of discussion. Mixed Methods Evaluation TIG views that mixing can occur at the level of inquiry purpose, philosophical assumptions, methodological design and/or specific data gathering technique. So, Mixed Methods Evaluation TIG sees our discussion can include the relationship between paradigmatic and theoretical lenses and methods.   The authors of this week’s posts will walk us through how the feminist lens informed inquiry purposes, choice of evaluation design, and methods.

 

Highlighted for FIE/MME week are:

  • Authors will explicitly talk about their worldviews, such as:
    •  what they believe in (they believe social justice),
    • issues they are concerned about (they are concerned with gender issues and marginalized populations),
    • and how their worldviews influenced evaluation questions they asked and their choice of methods.

Lesson Learned: Your evaluation lens is important. The feminist lens helps evaluators to see conflicting views of what the problem is.  With this understanding, evaluators consciously make decisions about what and whose evaluation questions to be asked.   The feminist lens also helps evaluators to see the diversity among a disadvantaged population.

Hot Tips:

  • Be reflective. You will also notice that evaluators are reflective of how they and their evaluations may be perceived by other people. They provide lessons learned from establishing relationships with evaluation participants, evaluation commissioner, and audience.
  • Match your analysis to your evaluation design. Evaluators decided data collection and analysis methods by considering evaluation questions, purpose of evaluation, and settings in which data collection took place. How to include perspectives of marginalized population is an important consideration for deciding methods.

Rad Resources:

The American Evaluation Association is celebrating the Mixed Methods Evaluation and Feminist Issues TIGs (FIE/MME) Week. The contributions all week come from FIE/MME members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

We are Silvia Salinas-Mulder, Bolivian anthropologist, feminist activist and independent consultant, and Fabiola Amariles, Colombian economist, founder and director of Learning for Impact. We have worked for several years as external evaluators for development programs in Latin America. The following ideas may help to operationalize the principles of gender- and human rights (HR)-responsive evaluation in complex, multicultural contexts.

Lesson learned: Terms of Reference (TOR) for an evaluation are not engraved in stone.

Tip: Reframe the often conventional evaluation questions and other aspects of the evaluation process to ensure that gender and HR issues surface, and evidence of change (or no change) in women’s lives is gathered. Take into account context-specific issues and gender dynamics, as well as relevant cultural patterns, such as the effects of migration in the family roles and decision-making processes within some agricultural community settings.

Lesson learned: Some stakeholders are tired of being interviewed, while others – especially rural women- are eager to be heard.

Tip: Be creative; evaluation techniques are the means not the end, and can thus permanently be created, recreated and adapted to each situation and context. For example, use “conversatorios” (round table discussions), as opposed to focus groups, to gather people with diverse backgrounds and perspectives to discuss over a particular issue of the evaluation; participants usually appreciate these reflective spaces and feel motivated to speak “outside the box”, while evaluators take a holistic overview of the topic.  Drawings, role plays and other popular education techniques may also facilitate participation of marginalized groups, including illiterate women.

Lesson learned: Answers to your questions may not contain key gender and HR issues to understand how change is occurring.

Tip: Awareness of specific cultural and gender communication patterns is crucial for an effective exchange. In any case, interviews should be dealt with as dialogues where people have the opportunity to express their priorities and points of view. Do not limit your interactions to a question-answer dynamic. Let people speak freely and “listen actively” to discover the essential. Respect and interpret the silences and do not insist on answers to your questions, rather focus on trying to understand the underlying meaning of each reaction. This will allow an eventual reconstruction of how change is occurring (Theory of Change) for the specific intervention and context, even if it has not been explicitly stated in the project design. Also, as evaluators we tend to focus on verbal communication, ignoring the importance of tone and gestures. Make sure you are alert to less explicit key messages.

Rad Resources:

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · · ·

I’m Dominica McBride, President of The HELP Institute, Inc. Recently, I wrote an article for New Directions for Evaluation on Sociocultural Theory (ST) and its applicability to evaluation.

Joke: There were a few young fish swimming near an older, wise fish and the old, wise fish says, “My, isn’t the water wonderful today?” and the young fish say, “Water? What water?”

Relevance: Culture is like water to fish – influencing our lives but often taken for granted or never even seen. It can be this ethereal and abstract concept. However, reflecting on and examining culture can be pivotal in personal and professional growth, program improvement, and societal change.

ST is one of the most comprehensive models on culture and human development, touching on all aspects of culture – the biological, psychological, interpersonal, linguistic, ecological, and historical. It provides simple guiding principles for evaluation and research practice:

Hot Tip: The unit of analysis is the sociocultural activity. Examine naturally occurring activities as it relates to the program or process (e.g., participants interacting). These activities should be considered in the context of the program, sociopolitical environment, family norms, organizational culture, etc.

Hot Tip: To understand a person, group, or social phenomena, we must ascertain the ever-changing environment and acknowledge and examine the development of the person, group, or program over time.

Hot Tip: Individual dynamics are affected by intrapersonal, interpersonal, and community dynamics – all of which are inseparable. So, to study a person or program, we must also consider the multifaceted influences that affect the participant behavior and the program.

Hot Tip: Groups have more variety within than between, which means there is more “cultural group” within a group and we must take this into consideration when learning of “another culture.” This fact can also help to dispel stereotypes. There are also many commonalities between groups. These phenomena help us to see and appreciate both the differences and links between us and others.

Hot Tip: Often times, we can get attached to methods. ST reminds us that the question should drive the methods and not the other way around. This assumption also encompasses the need for interdisciplinary work, opening our minds and hearts to other professionals and ways of doing things.

Hot Tip: In order to truly understand another, we must understand ourselves. Thus, we should take time to reflect on our development, cultural influences, personal and professional context, and intrapersonal dynamics. Without cleaning our own lens, we will always see others through tainted glasses.

Rad Resource:

Sociocultural Theory expert Barara Rogoff’s comprehensive book, The Cultural Nature of Human Development.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Hello! My name is Psyche Williams-Forson and I am an Associate Professor of American Studies at the University of Maryland College Park. Working under the themes of the cultures of everyday life and constructions of identity and difference, I study the material world through gender, food behaviors, and power. When asked some of the ways evaluators can grapple with the thorny terrains of power, I respond with lessons learned from a dinner party.

Several years ago, when I worked in Student Affairs, I participated in assessing our division’s plan for the inclusion and implementation of diversity vis-à-vis the university’s strategic plan. As one of three women of color on the committee I often left our meetings feeling marginalized. Shortly after the final report was submitted, I received an invitation “to celebrate” our committee’s work at the home of the Dean. As a junior administrator, I felt my attendance was more than requested—it was required. I responded affirmatively and was assured that I need not bring anything. When I arrived, I was greeted with the main dish—tofu kabobs! One of the other women of color took me aside and asked, “What is this?” My response was, in a word—“power.”

Lessons Learned: A natural evaluation of this scenario reveals that in both a professional and personal setting, power was enacted. On the committee, members of the group left meetings believing that no institutionalized processes of change were forthcoming. While a lot of rhetoric around diversity was exchanged few concrete examples of institutional accountability emerged. Simply put, “the diversity box was checked”: meetings were held, and people of color were in attendance. Short of this, little systemic action was proposed.

Lessons Learned: Never dismiss the subtleties of power and how they are exercised, especially in relation to privilege. Foods, like the celebratory tofu, mark time, place, and circumstances by their absence as well as their presence. The host of the dinner party perhaps thought she was exercising proper etiquette when she suggested that we not bring anything to dinner. Yet, the “elephant in the room” was the obvious discomfort most of us felt being forced into a situation where we were to eat bean curd.

Hot tip: Power and privilege work in silence(s). Be attentive to the ways in which evaluation can help to reconstruct or reshape power relations.

Rad quote: “Most of the choices we make are unconscious…We can participate in systems in ways we’re not aware of and help produce consequences without knowing it…both historically and in the present, without any intention to do so.” (Johnson, “Who Me?” http://www.agjohnson.us/essays/whome )

Rad Resource: Privilege, Power, and Difference, 2e, by Allan G. Johnson (McGraw-Hill, 2005)

This contribution is from the aea365 Tip-a-Day Alerts, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

· ·

Archives

To top