Greetings! I am Dr. Laura Peck, Principal Scientist at Abt Associates and Global Director for Abt’s Research, Monitoring & Evaluation Capability Center. My expertise is in the design, execution, and analysis of impact evaluations that leverage experimental and quasi-experimental designs to estimate impact that have a causal interpretation. I ask that those of you familiar with AEA’s rocky past with government prioritization of these so-called “rigorous” designs put that thinking to the side and be open to taking a fresh look at experiments.
All of the evaluations I engage in are, by necessity, both theory-based and mixed-method. This combination means better understanding the intervention’s implementation details, context, population, and so on—to explore what works, for whom, and under what conditions. I focus especially on subgroup analyses, because it’s often where the story lies.
How is a focus on equity showing up in my work? It is essential to integrate deeply and fully an equity focus into evaluations of whatever design. Doing so involves:
- being inclusive in teaming, engaging diverse voices and perspectives among researchers, participants, end-users;
- formulating equity-focused evaluation questions;
- being equity-focused with design, outcomes, and measures;
- collecting data with an equity focus;
- using nuanced analyses that reveal information needed to inform equity-related questions; and
- ensuring reporting and dissemination respond to varied stakeholder needs.
This changes the work I do by integrating diverse perspectives and practices into experimentation. It is for this reason I am especially looking forward to the Presidential Strand session at this Fall’s AEA conference. “The Dangers of Color-Blind Experimentation” (November 10, 10:15AM CST) promises to offer insights into how best equity principles and practices can enrich experimental evaluations, and how experimental methods can embed and advance equity. These two “e”s—experiments and equity—can operate in harmony. But, to do so, those who use experiments in their practice must acknowledge the history of unethical and dehumanizing research, rooted in systemic racism that has generated mistrust of this evaluation approach within non-white communities. By positioning those communities front and center, ensuring that research questions and methods are aligned, and coming to research with humility and an earnest enthusiasm to partner in the evidence-generation enterprise, I believe the future of experimental evaluations in practice can be strong, as we (re)shape evaluation together.
Rad Resources:
My 2020 book, titled Experimental Evaluation Designs for Program Improvement, was a step in debunking the myth that experiments are a blunt tool. You might recall my December 2016 AEA365 post on that point as well. In the book and the blogpost, I explain how experimental evaluation designs can be flexible to explore “what works” questions and inform program improvement efforts. Its second edition—or a future, companion book—must address the integration of equity principles and practices. Until then, I recommend the Kellogg Foundation’s Doing Evaluation in Service of Racial Equity resources. And, I am hopeful that this Fall’s AEA conference will provide food for thought on the practicalities of doing so.
Lessons Learned:
Many of the lessons I learn from being a recovering professor and now full-time practicing evaluator feed in to benefitting me in my other role, that of Co-Editor of the American Journal of Evaluation. Our field is massively diverse in so very many dimensions. We must demonstrate tolerance, humility, inclusiveness, and patience with one another as we work to advance our field with equity at the center of what we do.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.