Ed Eval TIG Week: Using Social Network Analysis to evaluate professional development for teachers by Valerie Ehrlich, Tim Leisman, Micela Leis, and Jeff Kosovich

Hello AEA 365! We are the Societal Advancement Insights & Impact team at the Center for Creative Leadership in Greensboro, NC. (Valerie Ehrlich, Tim Leisman, Micela Leis, and Jeff Kosovich.)

Professional networks among teachers play an important role in K-12 settings, where certain network characteristics have been linked to improved teacher practices and student outcomes. Building on Coburn & Russel’s 2008 work in Educational Evaluation and Policy Analysis, we use Social Network Analysis (SNA) to collect formative and summative evaluation data about program interventions to help teachers and administrators develop as leaders and improve performance.

Hot Tips:

1. Consider the rationale for using SNA: Does the program focus on creating relationships?

CCL co-designed and co-facilitated a year-long professional development program for teachers at Ravenscroft, an independent K-12 school in Raleigh, NC. The goals of the program were to improve teachers’ competencies in facilitative teaching practice for student engagement and to strengthen connections across school levels and within school divisions through a purposeful cohort design. We administered a social network survey at both the whole school and individual cohort levels in order to assess the relationship-building aspect of the intervention.

2. Consider a Pre-Post Design: How much will the network grow throughout the program?

Using a pre-post design is important to highlight the program’s effect. By using this in our work with Ravenscroft, we measured a notable increase in the teachers’ collaborative networks. On average, faculty reported 13 new collaborative connections each at the end of the program with colleagues they hadn’t collaborated with before the program – for a total of 1,635 new collaborative connections schoolwide!

3. Dig Deeper: What can you say about the types of network connections?

Ask about different ‘levels’ or ‘types’ of relationships: who do you share ideas with? Who do you collaborate with regularly? Who do you seek support from? This helps you speak not only to the quantity of relationships, but the nature and quality as well.

4. Examine Reciprocity: Are the connections superficial, or are individuals reporting the same relationships?

Compare measures of perceived connections with reciprocal connections. This allows evaluators to get an idea of how people perceive their networks as well as go beyond self-reporting to explore the nature of those relationships in practice.

5. Mind the Survey Length: SNA can provide amazing information, but it is a time investment.

SNA can be burdensome for large groups when the survey presents a large list of people. One way to ease that burden is to use survey display logic in the following way: First, ask participants to select the people they know from the list. Then, pipe those responses forward to ask about the nature/quality of those relationships. It helps make the survey less daunting!

Rad Resources:

• Do SNA in R! Contact our team at CCL for two tutorials about using R to conduct SNA.
• Learn more about our work with Ravenscroft here and feel free to contact us.

The American Evaluation Association is celebrating Ed Eval TIG Week with our colleagues in the PreK-12 Educational Evaluation Topical Interest Group. The contributions all this week to aea365 come from our Ed Eval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.