I am Susan Wolfe of Susan Wolfe and Associates, LLC. I am a community psychologist. My evaluation portfolio includes evaluating initiatives to reduce health and educational inequities rooted in systemic racism. Reducing inequities and systemic racism requires decolonizing white-supremacy based practices, including evaluation practices. My work requires a deep dive into my own place of privilege and how I approach evaluation, including the methods I use. As I engage in deep dives, I often go down many rabbit holes.
Hot Tip
Some rabbit holes are worth the detour. Continue to follow them.
One of the most recent rabbit hole trips began with a chapter by Guba and Lincoln (1994), followed by a book by Shawn Wilson (2008). Both provided a framework for me to think about the methods I use by examining the paradigms that guide my inquiry, and their relationship to colonizing practices. I will share what I learned from them here.
Rad Resources
- Guba, E.G., & Lincoln, Y.S. (1994). Competing paradigms in qualitative research.
- Wilson, S. (2008). Research is ceremony: Indigenous research methods.
The first paradigm is Positivism whereby evaluation relies on quantifying things and may incorporate experimental design. The evaluator acts as a “disinterested” party who informs policy and practice from a position external to the evaluation “subjects.” In this case, outcomes have been decided by “experts” often with token or no input from participants. The programs and outcomes are usually designed to change the participants without recognition of the context.
The second paradigm is Post-positivism which includes gathering data using qualitative methods and then rigorously analyzing it, often quantifying the results. The analysis is generally performed by people from outside the focal population who interpret the meaning through their own world experience. Use of most frequent themes may result in statements made by only one person being discounted – possibly the one statement made by someone brave enough to tell the truth.
Critical theory, the third paradigm, recognizes that evaluators need more reflective assessments and critiques of society and culture that consider power structures and oppression. Reality is more fluid than one fixed truth. Evaluation requires transactional dialogue between the evaluator and participants with the understanding that reality has been shaped by our cultural and other values.
The fourth paradigm is Constructivism, which recognizes that there is not one reality but there are many. Realities are specific to people and locations. Evaluators and participants come together to create common meaning.
Hot Tip
Examining the paradigm that underlies your methods is important because the paradigm influences the goal. The goal of positivism and post-positivism is to generate knowledge for the sake of knowledge and learning. The goal of critical theory and constructivism is to create change.
I am working to decolonize my evaluation practice. This includes working more closely with community members in a more participatory way consistent with critical theory and constructivism. Eliminating racial disparities requires addressing systemic racism. As a white person of privilege, it is important that the evaluative information I gather reflects the voices and perspectives of community members. Methods based in critical theory and constructivism paradigms help me to incorporate these voices.
Hot Tip
Evaluation is only worth doing and only ethical if the findings are used to improve the reality of the participants. The utility of the findings starts with your methods and the extent to which the voice of the participants is reflected in them.
Rad Resource
Smith, L.T. (2021). Decolonizing methodologies.
The American Evaluation Association is hosting Decolonizing Evaluation week. All posts this week are contributed by individuals committed to the decolonization of evaluation. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.
Shulha, L., & Cousins, B. (1997). Evaluation use: Theory, research and practice since 1986. Evaluation Practice, 18, 195-208.
Hello Susan,
Thank you for sharing your thoughts. The sentiments shared in your post really resonated with me. As a Masters of Education student (with previous coursework in Psychology), currently taking my first course in evaluation, I appreciated the way you considered values, collaboration and community and how it impacts an evaluation. In my course, I recently reviewed articles that imply that more recently there has been an emergence of “collaborative modes of evaluation. Unlike strictly consultative approaches, collaborative models tend to aspire to more equitable power relationships between evaluators and program practitioners leading to jointly negotiated decision making and meaning making” (Cousins & Shulha, 1986, p. 200). This supports your point about reality being shaped by one’s cultural and other values. Your statement that there are many realities and that they are specific to people and locations, only strengthens the argument that the evaluation process must be collaborative, and long gone are the days of a rigid evaluator who is completely disconnected from the community that their evaluation is intended to serve.
I appreciate your thoughtful insights!