Internal Evaluation TIG Week: Strengthening Learning Culture and Building Evaluation Capacity Before Measuring Impact by Rafiqah Mustafaa

Hello! I’m Rafiqah Mustafaa, PhD., and I serve as the Assistant Director of Learning and Improvement at the Collaborative for Academic, Social, and Emotional Learning (CASEL). In this blog, I detail how our internal evaluation team (Dr. Ally Skoog-Hoffman, our Senior Director, Research and Learning, and I) has prioritized strengthening our organization’s learning culture and evaluation capacity so we can effectively conduct internal evaluation.

 As a nonprofit organization that advances social and emotional learning (SEL) research, practice, and policy to build equitable learning environments for young people, we constantly ask, “are we having the impact we hope to have?”

To measure that impact, our internal evaluation team has begun supporting leadership and staff with

• Defining our shared goals,
• Conceptualizing team-level outcomes aligned to our shared goals,
• Updating metrics and data collection tools, and
• Building a shared data management system for visualizing and using data for learning and decision-making.

 Along the way, we identified the importance of attending to our organization’s evaluation capacity and learning culture so we can conduct useful internal evaluations. Here’s a rad resource, hot tip, and cool trick that might be useful to others engaging in similar efforts.

 Rad Resource:

To design a useful and data-informed evaluation capacity-building effort, we first assessed our evaluation capacity using a staff survey based on the Evaluation Capacity Assessment Instrument from Taylor-Ritzler et al. (2013). We measured organizational, departmental, and individual knowledge, beliefs, and practices related to evaluation and identified areas of strength (e.g., staff valuing the importance of evaluation) and areas of growth (e.g., limited opportunities to reflect on data on initiative effectiveness across departments). After conducting the assessment, we facilitated data reflection sessions with staff and leadership teams and highlighted how the perspectives they shared on the survey will inform the development of our org-wide internal evaluation system.

Hot Tip:

Prioritize professional learning for staff as a key internal evaluation practice. In designing our new internal evaluation system, we envision non-evaluation staff engaging in key evaluation activities—articulating how their team’s work aligns with organizational priorities, defining metrics and goals, and feeling confident with using data for decision-making. To support staff’s engagement with these practices, we’ve offered professional learning sessions at our leadership team retreat, all-staff retreat (see Figure 1), and respective team meetings. Later this year, we’re launching an internal data and learning professional learning community (PLC) comprised of one member from each department. This PLC will gather to develop evaluation knowledge and skills, navigate problems of practice, and serve as liaisons between our internal evaluation team and their departments.

Cool Trick:

Draw on common language to socialize around internal evaluation. SEL is at the core of all of CASEL’s work. And since SEL is critical for collaboratively conducting evaluation, we’ve begun to leverage this connection. When introducing an evaluation topic to staff in a professional learning session, we ask them to reflect on the relationship between SEL competencies and constructs like identity, social awareness, and curiosity and the practice of learning about and improving their work. This reflection creates the space for staff to draw on their own social and emotional competencies (see Figure 2) to build confidence with engaging in internal evaluation. 

We hope this resource, tip, and trick are useful to other internal evaluators supporting organizations in building their capacity to measure the impact of their work!

---

The American Evaluation Association is hosting Internal Evaluation (IE) Topical Interest Group Week. The contributions all this week to AEA365 come from our IE TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.