Hello, AEA365 community! Liz DiLuzio here, Lead Curator of the blog. This week is Individuals Week, which means we take a break from our themed weeks and spotlight the Hot Tips, Cool Tricks, Rad Resources and Lessons Learned from any evaluator interested in sharing. Would you like to contribute to future individuals weeks? Email me at AEA365@eval.org with an idea or a draft and we will make it happen.
We are Lenka Berkowitz from Alliance for a Healthier Generation (Healthier Generation) and Elena “Noon” Kuo from Kaiser Permanente’s (KP) Center for Community Health and Evaluation. We collaborate to evaluate a partnership between Healthier Generation and KP, which is one branch of the Thriving Schools Initiative. Thriving Schools aims to advance equitable, whole-child health by supporting policy, practice, and environmental changes in schools and districts across eight regions of the United States.
As part of our evaluation process, we used meaning-making sessions as a tool to actively include program staff in interpreting evaluation findings and brainstorming action steps, leading to a deeper and broader understanding of program impact and buy-in for taking action. Meaning-making, grounded in reflective practice and utilization-focused evaluation, is an important part of the feedback loop.
It promotes deeper understanding and shared learning among key players, such as staff and leadership, who may hold different perspectives on the findings. Throughout this process, they can reflect on different beliefs, assumptions and life experiences as they relate to the findings and agree on data-informed program improvements. The meaning-making process begins with the evaluator taking the lead on data analysis; however, the session planning and facilitation should be a close collaboration with program staff. The session is followed by action planning, where staff collaboratively prioritize recommendations for implementation based on the group’s collective insights.
Lessons Learned
Embrace the Iterative Approach: Try something that you think will work, then keep iterating on it. Avoid creating the perfect process the first time through, which is a recipe for frustration. Instead, be flexible, responsive, and allow the work to evolve over time to best suit your needs.
Time, Intentionality, and Feedback: It’s worth taking time with each step of the process, from design to the session itself. Intentionality about process, setting goals, discussing plans during the session, generating buy-in, and asking for feedback throughout the process will help you have a more successful and value-added meaning-making session.
Balance Insights with Action: There is a delicate balance between meaning-making and action planning. People often rush into action planning, especially if they aren’t used to reflective practice. Facilitators may need to guide people between “so what” (what the data means) and “now what” (what are next steps).
Be Mindful of Other Factors at Play: Several factors impact the success of your meaning-making session, many of them out of your control! Organizational culture, existing power dynamics, differing understanding of the role of evaluation, the quality and complexity of the data, and more. Be aware of these factors, and when possible, acknowledge them and support conversations that drive towards actionable steps rather than dwelling on impossible wishes.
Cool Trick: Reflection Questions
Thoughtful questions focused on interpretation, reflection, and context can prompt richer discussion. Sample questions: ?
- What surprises you in the data?
- What confirms/contradicts your assumptions?
- How could implementation be improved?
- Roses: What are the strengths?
- Thorns: What are the challenges?
- Buds: What are the opportunities?
- Fertilizer: What are the lessons? What do we need to do?
Cool Trick: Predictions about Data
Before staff review the data, have them spend 10 minutes writing down predictions about what the data will say. This exercise helps surface beliefs, assumptions, and biases that may otherwise remain unconscious.
Rad Resource: Technology for Engagement
Use collaborative technology like Mentimeter or Mural to actively engage staff in meaning-making. For example, small groups can post insights on virtual sticky notes, then collectively categorize and prioritize themes. Interactive exercises build shared understanding.
Rad Resource: Summary
We created a meaning-making one-pager based on our experiences.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.