Hello, AEA365 community! Liz DiLuzio here, Lead Curator of the blog. This week is Individuals Week, which means we take a break from our themed weeks and spotlight the Hot Tips, Cool Tricks, Rad Resources and Lessons Learned from any evaluator interested in sharing. Would you like to contribute to future individuals weeks? Email me at AEA365@eval.org with an idea or a draft and we will make it happen.
Hello! We are David Fetterman and Paul St. Roseman. David Fetterman is a past president of the American Evaluation Association and the president of Fetterman & Associates, an international evaluation consulting firm. He is also on the faculty of Claremont Graduate University and the Pacifica Graduate Institute. Paul is a former student of David Fetterman at Stanford University. He is the founder and owner of Data Use Consulting Group, an independent evaluation agency, and the Chief Information and Technology Officer at Reach University.
Paul and David are currently collaborating on the empowerment evaluation of The Just Trust, a $350 million initiative funded by Chan Zuckerberg to reform the criminal justice system.
Our Approach
We are using two connected approaches to guide this work:
1. Empowerment Evaluation
- Purpose: Give staff, partners, and communities the tools to monitor and improve their own work.
- Principles: Improvement, community ownership, inclusion, democratic participation, social justice, and evidence-based strategies.
- How it Works:
- Establish shared mission and goals.
- Take stock of current efforts and performance.
- Plan for the future with clear metrics and strategies.
- Use ongoing feedback loops to make adjustments.
- Establish shared mission and goals.
2. Collaborative Data Analysis
- Purpose: Bring staff, partners, and stakeholders together to make sense of data and decide what to do next.
- Core Practices:
- Shared responsibility for reviewing, interpreting, and using data.
- Clear norms for respectful, inclusive discussion.
- Turning insights into actionable strategies.
- Shared responsibility for reviewing, interpreting, and using data.
Benefits: Builds trust, strengthens decision-making, and makes data a shared resource.
Embedding MEL in The Just Trust
Our MEL work is shifting from occasional, separate evaluation exercises to an ongoing, embedded process.
That means:
- Standardized metrics and reporting across grantees and initiatives.
- Quarterly milestones to track progress and adjust strategies.
- Learning-oriented evaluation—not punitive measures.
- Shared ownership across all teams.
Cool Trick: How AI Supports MEL
The Just Trust uses AI tools to make MEL more efficient, inclusive, and accessible through the use of:
ChatGPT: Use ChatGPT and create your own GPT with your documents to improve the quality of output
Notebook LM: Transform documents and URLs into engaging podcasts; Summarize reports, identify themes, and validate content.
Memorial Article (PDF) Transformed into a Podcast
Napkin AI: Create infographics and visual summaries.
Visual Highlighting Pros and Cons of Existing Evaluation Design
Claude AI: Inquiry, summarize data, create new knowledge, format web-ready materials for sharing.
Rural Senses: Gather community voice data—even in rural or offline settings—and turn it into actionable insights.
- Identifies relevant references
- Generates a visual theory of change
- Generates a draft report
Benefits
- Speeds up analysis.
- Maintains consistent messaging.
- Creates accessible visuals for staff, grantees, and donors.
Safeguards
- All AI outputs are reviewed by humans for accuracy and context.
- Sensitive data is protected under GDPR-compliant systems.
Rad Resources
- Empowerment Evaluation: Knowledge and Tools for Self-Assessment – David Fetterman’s practical guide.
- Empowerment Evaluation and Social Justice – David Fetterman’s demonstration of how to use evaluation in pursuit of social justice
- Artificial Intelligence, Healthcare, and Evaluation: Altering the Landscape – David Fetterman and David Fetterman II – Their chapter about how artificial intelligence has altered the healthcare and evaluation landscapes
- Mind-blowing AI-generated Podcast: A Conversation about Michael Scriven and David Fetterman (Fighting it Out) by David Fetterman – David Fetterman’s explanation of how to use AI to transform a document into a podcast.
- Artificial Intelligence (AI): Using AI to Build Evaluation Capacity by David Fetterman – David Fetterman’s discussion and illustration of how to use AI to create images
- Ray Ban Meta Smart Glasses: Pictures, Videos, Music, and Translation – David Fetterman, Paul St. Roseman, and David Fetterman II’s demonstration of how to use AI “smart glasses” to enhance evaluation
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.