My name is Matodzi M Amisi and I am a research associate with CLEAR-AA. I have a keen interest in evaluation and evidence use within the violence prevention sector. Since 2016, I have actively participated in the Violence Prevention Forum, which aims to foster collaboration between the government, academia, and civil society in generating evidence on effective strategies to prevent violence in South Africa. It is through this experience that I have come to understand the significance of problem questioning in our efforts to decolonize evaluation practice.
A few years ago, I came across Tom Archibald’s seminal paper exploring Bacchi’s feminist “What is the problem Represented to be” (WPR) problem questioning method. The paper expressed ideas that aligned with what I had observed as an evaluation manager in the South African government and with non-government organizations delivering programs. Evaluation treated programs as technically bounded, intervening to address objectively identified (self-evident) undesirable social conditions. The focus of evaluation was to assess whether the intervention worked or not, rarely engaging with the questions of whether the problem the intervention was addressing was the correct one and considered important by the people whose lives the interventions were meant to transform.
Bacchi’s views resonated with me. She argued that problems are socially constructed. This is not to say that there are no undesirable social conditions, but rather that “problems” do not exist outside of politics and policy processes, waiting to be solved. Instead, they are produced within specific sites and contexts of existing power relations. Those of us who are educated and have both the resources and privilege to observe society and study it often exercise great influence in determining and framing social situations as problems and determining what is needed to address them. We often do this without questioning our own assumptions and without considering the beliefs and thoughts of the communities we intervene in. I believe this is the fundamental problem that efforts to decolonize evaluation practice aim to address.
Here, Bacchi offers us a tool to question ourselves. The WPR invites us not to take a problem as given, but to ask: what is the problem represented as? What assumptions underpin the representation? How did the representation come about? What has been left unproblematic? Can the problem be thought of differently? What effects has the representation produced? And where and how has the representation been reproduced? By systematically answering these six questions, evaluators can question, probe, and problematize modes of thinking that shape interventions, uncovering which voices are heard and which are not in framing the problem and determining what is done to address it. This, I believe, is at the heart of decolonized evaluation science.
Systematic problem questioning enables decolonial evaluators to go beyond just changing methods and approaches to evaluating programs, even though this is an important part of it. By using a method that is conscious of how power shapes what is considered true or real, evaluators can go beyond metaphorical decolonization of evaluation. In doing so, they can challenge (or at least identify) hierarchies of power that shape programs and potentially reclaim space for those who are marginalized to have a voice in conceptualizing social problems and resulting interventions.
I saw this in my own work. Two years ago, we applied the WPR approach to assess how the problem of violence against women and children was represented in evaluative research in South Africa. The analysis revealed how little consideration was given to the values and beliefs of communities in the representation of the problem, and the heavy influence of international donors, international development agencies, global North universities, and academic paradigms. Consequently, a fragmented understanding of the problem had resulted, and interventions were often irrelevant and the results not sustainable once the program ended.
Lessons Learned
Through this experience, I learned that the process of decolonization requires us to shift which knowledge shapes the understanding of a problem and what is considered successful interventions. I also learned that truly transforming practice requires leadership from individuals who are willing to relinquish their power to determine what is of value and what is not. It is a brave ask.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.