This is Vidhya Shanker, a doctoral student at the University of Minnesota. At Evaluation 2010, I served as a scribe for Roundtable Presentation 111, Maximizing Our Collective Talent: Conversations with Senior Evaluators, which brought evaluation leaders together with novice and mid-career evaluators to engage in evaluation discourse around 8 different issue tables. I chose it because sessions more typically portray people of color and indigenous people as participants—and increasingly staff—of programs being evaluated rather than as evaluation leaders.
Lessons Learned: These are from one table facilitated by Katherine Tibbetts on women and evaluation and another by Stafford Hood on culturally responsive evaluation:
- The Great Society benefited middle-class, white women who gained employment in social work programming. Women constitute the numerical majority and some of the most recognizable names in evaluation. Considered in context, however, the “caring” fields (education, health, and social services) that feed into evaluation are majority-female, and yet the most recognizable names remain male.
- Intersectionality refers to the entanglement of multiple, overlapping, sometimes conflicting matrices along which we all experience power/ privilege/ entitlement as well as marginalization. Whether approached through race/ ethnicity, gender/ sexuality, and/ or another entry point, evaluation involves self-knowledge, critical thinking, and reflexivity.
- Language can perpetuate or disrupt existing power dynamics and stereotypes. For example, we tend to objectify segments of the population (e.g., “single mothers”). Those exhibiting symptoms of social problems are often pathologized and treated as “the problem.” Similarly, we operate on assumptions of freedom, rationality, and individual choice, although local and global contexts and histories open and constrain all our choices. Even the conceptualization of a question is a political act, as is how that question is subsequently framed and investigated.
- Working toward social justice involves understanding the political implications of our actions and paying attention to consequential validity. To what extent, for example, do we respect separate spaces? To what extent do we validate relational ways of knowing?
Great Resources:
- Hood, S., Hopson, R. K. & Frierson, H. T. (2005). The role of culture and cultural context: A mandate for inclusion, the discovery of truth and understanding in evaluative theory and practice. Greenwich: IAP.
- LaFrance, J. & Nichols, R. (2010). Reframing evaluation: Defining an Indigenous Evaluation Framework. Canadian Journal of Evaluation, 23, 13-31.
Hot Tip: Espousing critical theory (see New Directions for Evaluation 127), and certainly identifying as a member of a marginalized group, provide no guarantees for conducting feminist or culturally competent evaluation. For example, being in positions of relative power, many evaluators attribute resistance among program staff and participants to their categories as “false consciousness” instead of entertaining the possibility that this refusal may be a legitimate critique of and resistance against the categories themselves.
At AEA’s 2010 Annual Conference, session scribes took notes at over 30 sessions and we’ll be sharing their work throughout the winter on aea365. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Dear Yasmine–
So sorry I’m just now seeing your comment–and entirely by accident–more than a year after you posted it. Blog posters are not notified when someone comments on our posting specifically and I must have missed when your comment was sent out to subscribers of AEA365 comments more generally.
Anyway, the connections you draw are insightful, and nearly 10 years after posting this, I continue to be most inspired by decolonizing and indigenous frameworks, which name the quest toward full sovereignty, emancipation, and liberation (political, economic, cultural, spiritual, environmental, etc.) as underlying research and evaluation work
I love the question you pose and hope that you continue to write about them and other connections.
I found your blog on the AEA365 website titled, Scribing: Vidhya Shanker on Lessons Learned from Evaluation Leaders an interesting read.( http://aea365.org/blog/?s=entitlement&submit=Go)
The title itself drew me in as I firmly believe that we learn from others as we all see things through different lenses. Some of her “lessons learned” got me thinking about how in general an evaluation program takes place. Since we all see something from a different point of view, we all too must evaluate differently. Shanker, (2010) additionally goes on to state “that we often objectify segments of the population and those exhibiting the social problem are often pathologized as the problem, her example being single mothers.” So how do we identify what the problem really is in a social context?
Your “hot topic” discussion proposes “being in a position of relative power, many evaluators attribute resistance among program staff and participants to their categories as “false consciousness” instead of entertaining the possibility that this refusal may be the legitimate critique of and resistance against the categories themselves.”
This statements links well with the article provided Reframing Evaluation: Defining an Indigenous Evaluation Framework. The article elaborates on the use of evaluation regarding, in this case, the native population. This article points out how there is often a close connection between research and evaluation, with certain populations, which can often lead to cultural exploitation and the loss of intellectual rights (pg14). The article further goes on to suggest that evaluation is often taught in a western tradition limiting the processes used (pg 18), and this alone can be intimidating for some target groups.
I enjoyed the summarization laid out in your article for developing fundamental principles of the Indigenous way of knowing, that should an evaluation should incorporate. However, this article brought up some questions such as:
What type of results will we get if we do not have full participation with our target group?
How does one ensure all individuals feel respected during an evaluation process of mixed ethnicity?
How often do we see evaluation as an attempt to check what we are doing right, rather than just learning opportunities and how can we change that general perspective?
Yasmine
References
AEA365 (2010), Scribing: Vidhya Shanker on Lessons Learned from Evaluation Leaders. Retrieved July 22 2017 from http://aea365.org/blog/?s=entitlement&submit=Go
LaFrance, J. & Nicols, R. (2010). Reframing evaluation : Defining an Indigenous Evaluation Framework. Canadian Journal of Evaluation, 23, 13-31.