ICCE TIG Week: Invisibility in Evaluation by Stephen Porter

I am Stephen Porter, an evaluator in Oxfam, an international non-governmental organization (INGO). A major part of my job is to provide evidence on whether people live equitably and free from the injustice of poverty.  When conducting evaluations we often seek to apply tools to improve the quality and use of evaluations in an ethical manner. Although we work to well-developed standards (such as the JCSEE Program Evaluation Standards and the OECD Quality Standards for Development Evaluation), we do not tackle the issue of invisibility head-on.

In evaluation, the perspectives of the people we are meant to serve remain invisible and left behind if we have no data about them. This is a failure of perspective rather than a technocratic oversight. In thinking about my own experience of invisibility in evaluation, for a recent discussion at the DC consortium student’s conference, I reflected on three novels:

• J.M. Coetzee’s Waiting for the Barbarians;
• George Orwell’s 1984;
• Ralph Ellison’s Invisible Man.

An allegory for apartheid South Africa, Waiting for the Barbarians focuses on a nameless male character working through his own complicity in cruel acts and injustice. In this book the invisibility is of ‘the other’. The Barbarians remain on the periphery of the story rarely encountered except in situations of subjugation, snatched sightings and hearsay. In evaluation processes there is also often a group on the periphery that is known, but does not participate as stakeholder or is ignored in data collection and analysis. The remedy often suggested is to be ‘more participatory’. Yet, the issue in the evaluation might not be of neglect of participation, but of power that prevents participation. Some people are invisible in evaluations because they are a perceived threat, neglected or they compete for resources.

In the novel 1984, double think, the acceptance of competing contradictory beliefs, is a mechanism for invisibility: War is peace, ignorance is strength, and freedom is slavery. Sometimes reading evaluation reports conducted in authoritarian states, truth can become invisible. Ethnic tensions that simmer beneath the surface are not mentioned and the line of the ruling party is represented without question.

Invisible Man is a story of racism in America. A man is invisible because of the colour of his skin. This form of invisibility comes into play when, as an evaluator, you glimpse or you cannot even see a phenomenon. A social asset in a community that enables resilience is ignored, an organizational practice that puts children in harm’s way is misrepresented. People are invisible because they cannot be seen, even when they are in front of you.

As evaluators working in complex international settings, we need to recognize that the systems we have built and standards we apply for evaluation practice do not always sufficiently value the voice and perspectives of populations left behind. While they remain invisible we cannot effectively overcome injustice however, it is perhaps our job, to make these voices more visible through our reporting mechanisms.

Rad Resource: “Leaving no one behind in our evaluation practice”

The American Evaluation Association is celebrating International and Cross-Cultural (ICCE) TIG Week with our colleagues in the International and Cross-Cultural Topical Interest Group. The contributions all this week to aea365 come from our ICCE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.