Hello! We are Rebecca Teasdale (University of Illinois Chicago, USA), Jennifer McNeilly (University of Illinois Urbana-Champaign, USA), and María Isabel Ramírez Garzón (Pontificia Universidad Javeriana, Colombia). We are excited to share key takeaways from a research study we recently published with Judit Novak (University of Oslo, Norway) and Jennifer Greene (University of Illinois Urbana-Champaign, USA) in the American Journal of Evaluation.
Our study focused on values, which are beliefs about what is important or desirable that guide people’s decisions and behavior. The AEA Guiding Principles and the Program Evaluation Standards call on evaluators to identify and articulate the key values that shape evaluation studies. Yet there has been little empirical research about values in evaluation practice. As a result, little is known about evaluators’ values, how those values shape the evaluation process, and how evaluators negotiate conflicting values. We set out to learn more by conducting semi-structured interviews with eight experienced North American evaluators. We inductively analyzed the interview transcripts to surface the values they held and how those values “showed up” in their evaluation practice. Here, we share Lessons Learned, Hot Tips, and Rad Resources drawn from our study.
Lesson Learned:
Values show up in every component and all stages of the evaluation process. Evaluation is sometimes portrayed as neutral and value-free. Yet, our study found values permeated decisions throughout the evaluation process. For example, valuing inclusion motivated some evaluators to seek out the perspectives of policymakers, program administrators, staff, and participants and incorporate those values into evaluations. Those who valued collaboration partnered with program staff when developing logic models and evaluation questions, and members of the focal community as co-evaluators. Valuing systematic, high-quality empiricism motivated evaluators to use multiple methods, employ instruments that were valid for the purpose and context, and strive for rigor (even in challenging contexts).
Lesson Learned:
Values shape how evaluators position their work. Evaluators’ values also shaped their stance toward evaluation overall. For example, valuing equity and social justice motivated some evaluators to focus their practice on programs that addressed critical social issues and supported communities that have been historically marginalized. Those who valued educating stakeholders positioned their practice to foster learning, rather than focusing strictly on accountability. Values related to making things better in the world motivated evaluators to make a difference for their clients, program participants, and society at large.
Hot Tip:
Our findings can support evaluators in identifying and articulating their own values. We identified and examined 12 values that shaped the work of evaluators in our sample, some of which we described here. We invite evaluators to use our findings as a starting point for self-reflection to better understand their own values and how those values guide their choices and behaviors in evaluation practice. Our study also provides empirical examples of how evaluators navigate conflicting values, which we hope will help evaluators manage values conflicts that arise in their work. In sum, we hope our study will support evaluators in identifying and discussing the values dimensions of evaluation practice.
Rad Resources:
This study built on exciting work by other evaluation scholars. We learned a lot from the following resources and encourage evaluators to check them out to learn more about values in evaluation practice:
- Editorial in the Evaluation Journal of Australasia (discusses what is meant when we talk about values), by Hassall, Gullickson, Boyce, & Hannum
- Evaluating and Valuing in Social Research, book by Schwandt & Gates
- Making Values Explicit in Evaluation Practice, article by Gullickson & Hannum
- Findings from an Empirical Exploration of Evaluators’ Values, research study by LaVelle, Stephenson, Donaldson, & Hackett
The American Evaluation Association is hosting Research on Evaluation (ROE) Topical Interest Group Week. The contributions all this week to AEA365 come from our ROE TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.