Hi, I’m Barbara Klugman. I offer strategy support and conduct evaluations with social justice funders and NGOs in South Africa and internationally. I practice utilization-focused evaluation, frequently using mixed methods including outcomes harvesting and social network analysis (SNA). My own history spans social activism, directing NGOs and both working for and being on boards of foundation.
AEA’s Conference theme is Speaking Truth to Power, something that is particularly challenging because of inequitable power relations between nonprofits and their funders, and even between boards and staff. Evaluators can play a useful intermediary role by providing both the evidence and the facilitation to open space for honest communication.
Hot Tip: I have found the following six factors influenced the effectiveness of my communication across power divides:
- Timing of the evaluation and a formative or developmental approach may enhance both grantee and funder interest in the outcomes.
- Making learning rather than compliance the evaluation objective creates an environment that welcomes insights to strengthen effectiveness and removes much of the fear and risk from evaluation.
- The evaluator needs a substantial capacity for evaluation practice that enhances trust-building to undercut anxiety and establish rules of engagement that allow those with least power the ability to engage, influence and use findings.
- The production of high quality evidence while self-evident will be more effective in speaking truth to power if all parties have agreed on the questions, mix of methods and evaluation rubrics
- A commitment to and comfortableness with the role of evaluator as social justice advocate assumes that the evaluator can navigate when it is appropriate for her to speak, and when to empower the evaluand to do so.
- Terms of reference give the evaluator the independent right and resources to communicate findings to audiences beyond the intended users or those to whom they disseminate findings. While recognising the concomitant ethical responsibility to do no harm, the right and resources to publish findings is critical to the ability of an evaluator to speak truth to power and for the resources that go into evaluation to contribute to broader learning in the field.
Rad Resources: As an illustrative example, see the public communications from the evaluation team of the Ford Foundation’s $54m Strengthening Human Rights Worldwide global initiative. The ToR included funds for the team to publicize findings in Spanish and English which included the summary report, a series of blogs and videos, an article for the international human rights journal SUR and a reflection in Alliance magazine.
Blogs:
The Value of Diversity in Creating Systemic Change for Human Rights
Finding Equity – Shifting Power Structures in Human Rights
Addressing Systemic Inequality in Human Rights Funding
Videos:
The Human Rights System is Under Attack – Can it Survive Current Global Challenges?
The Changing Ecology of the Human Rights Movement
Funding an Effective Human Rights Movement
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hello Ms. Klugman,
My name is Shauna Jeyaseelan. I am currently completing my Professional Masters of Education program at Queens University and am enrolled in a program evaluation course which is why I was instantly intrigued by your post. It seems that your experiences in social activism, directing NGOs, and administrative roles in various boards have given you rich experiences to provide such insight into effective evaluative practices. Thank you for sharing your thoughts.
I appreciate the practical tips you have provided which I see can be applied within the school structure I teach in. I have witnessed firsthand the affects of timing of evaluations. I have seen standardized testing implemented in my school 3 times a year which are an evaluation of the student’s growth academically. Per testing period there are 3 tests students take (a total of 9 all year) which are multiple choice approximately 2 hours per test. These testing periods are scheduled quite close to one another and I find that because these are so frequent and demand immense mental output from the students – they lose motivation to put forth their best efforts and become more apathetic towards their results.
I feel the production of high-quality evidence all depends on the objectives, the driving questions, and methods selected that best address the question posed. I like that you also mentioned the role of the evaluator to ‘empower the evaluand’ to be involved in the process of change. Having high quality evidence and not knowing where to begin in terms of the process of change can be frustrating without the help and guidance of the evaluator. I am most curious about the publishing of findings of a company to a broader audience and how that affects the relationship between the evaluator and the evaluands, the frequency of future evaluations, and how this would not cause harm but be beneficial from the company perspective. Knowing that making these findings available to the public/their reputation on the line would be an outcome, what would motivate companies to be open to evaluations?
Hello Ms. Klugman,
I am a student completed a professional master’s of education. I am currently taking a course on program evaluation. Much of the learning has been done through an educational lens, and is centred around organizational improvement. After reading your post I am inspired that program evaluation can be used to speak truth to power, and implement social change.
I find it incredibly interesting how in the world of non-profits and NGOs, there is so much attention paid to how best navigate the delicate power balance between the funders, the organization, and the evaluator. This is the first time that I have considered the evaluator to be more than a neutral body in the process of evaluations, which I have found in theory to be quite prescriptive. The role you describe here where the evaluator can assume the role of a social justice advocate is inspiring.
Thank you for providing a different perspective to what my (limited) understanding of program evaluation is. I will look forward to continued learning in this field knowing that there are different opportunities for evaluators to make a difference in the world.
Brandie