Hi, I’m Xiaoxia Newton, a member of the 2019-2020 Guiding Principles Working Group and an Associate Professor at Cato College of Education, University of North Carolina Charlotte. In this blog, I share what my colleagues and I learned about the tension between evaluators actively engaging program stakeholders in the evaluation process and evaluators’ obligation to abide by the AEA Ethical Guiding Principles(GP ).
The GP and our empowerment evaluation
The AEA GP “are intended as a guide to the professional ethical conduct of evaluators.” While emphasizing the interconnectedness of the principles, AEA acknowledges that these principles might even conflict with each other. This sentiment resonated with the experiences my colleagues and I shared while implementing an empowerment evaluation in order to enhance public school leadership capacity for evidence-based decision making of the program that we were evaluating.
We chose the Empowerment Evaluation (EE) framework to guide our evaluation work because of the fit between the program design and the key features that characterize EE. The program attempts to address a complex social problem and therefore adopts a whole-child approach that engages multiple community members and is at the very beginning stage. EE focuses on improvement and empowerment, emphasizes collaboration between evaluators and stakeholders, and employs both quantitative and qualitative methods. Given the program design, its context, and its stage, EE offered an ideal framework guiding our evaluation effort. But as we know, empowerment evaluation is not without controversy (see e.g., Stufflebeam, 1994; Scriven, 1997)
From our perspective, these tensions surrounding empowerment evaluation seem to stem partly from the inherent tension within the Guiding Principles. Take a look at the following four Guiding Principles:
A1. Adhere to the highest technical standards appropriate to the methods….
C4. Assess and make explicit the stakeholders’, clients’, and evaluators’ values, perspectives, and interests …
E1. Recognize and balance the interests of the client, other stakeholders, and the common good…
E5. Mitigate the bias and potential power imbalances that can occur as a result of the evaluation’s context. Self-assess one’s own privilege and positioning…
In our view, principles A1/E5 and C4/E1 can lead to different interpretations in evaluation practice depending on evaluators’ philosophical and methodological training and preference.
In our own empowerment evaluation, we struggled with balancing the fine line between being external evaluators aiming to hold “the highest technical standards” (A1/E5) while aspiring to make evaluation a meaningful and useful activity for program stakeholders and ultimately empower them to do self-evaluation (C4/E1).
Lessons Learned: Being cognizant of GP will encourage reflection on our role as evaluators
The GP provide an excellent opportunity for examining the assumptions and values evaluators embody when designing and conducting evaluations.
The 2020 AEA annual meeting has emphasized how the GP are reflected in presentations proposed by members. As evaluators, we can be explicit about how we embed GP in our evaluation work and what struggles we face as we practice ethical evaluation and abide by the Guiding Principles.
This post is part of an occasional series on the AEA Guiding Principles. Each post in the series was contributed by a member of the AEA Guiding Principles Working Group. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.