AEA365 | A Tip-a-Day by and for Evaluators

May/11

6

EdEval Week: Shelly Engelman and Tom McKlin on assessing the evolution of Social Networks Using NodeXL

Hello all! This is Shelly Engelman and Tom McKlin, and we are evaluators at The Findings Groups, LLC, a privately-owned applied research and evaluation firm with a focus on STEM education. Recently, we have used NodeXL, a free and open add-in for Excel, to conduct a social network analysis to assess inter-organizational collaboration among a group of teachers. The goal of the program is to build a vibrant, supportive community of computing teachers from high schools and universities across Georgia. Our client was interested in the following evaluation questions: What partnerships are forming between which organizations? What is the strength of those partnerships? How do these partnerships change and evolve over time?

Using NodeXL, we were able to visually document the evolution of collaboration among computing high school and university teachers across time:

Figure 1 (on the left) shows only those participants in the program who have collaborated with other program participants. It shows that College M is a hub directly connecting three other colleges and one high school. Figure 2 (on the right) shows the partnerships that have grown as a result of 8 months of program participation. The participants are responding to the programmatic intervention in ways that promote collaboration. These two figures help answer the first evaluation question (what partnerships are forming between which organizations?) and the third question (How do these partnerships evolve over time?). We might use line thickness to answer the second question (What is the strength of these relationships?). For example, a thick line would show a strong connection, and a thin line would show a weaker connection.

Hot Tip: NodeXL quantitatively provides users with density scores (or number of connections between members), centralization indices (which assess information exchange), and clustering coefficients (which identifies clusters within a social network). In the above two sociograms, density scores were higher at 8-months than at baseline and centralization indices show a reduction in the hierarchical structure over time.

Future Consideration: Social Network Analysis using NodeXL not only assesses social relationships within a network, but also may be used to capture and identify changes within and across networks. An interesting next step may be to develop and test theories regarding how networks change, evolve, adapt, and decay. A new book edited by Alan J. Daly highlights how social networks evolve to create reforms in education: http://www.hepg.org/hep/book/131/SocialNetworkTheoryAndEducationalChange

Rad Resources:

The American Evaluation Association is Educational Evaluation Week with our colleagues in the PreK-12 Educational Evaluation AEA Topical Interest Group. The contributions all this week to aea365 come from our EdEval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

3 comments

  • Monitoring and Evaluation NEWS » Blog Archive » AEA resources on Social Network Analysis and Evaluation · April 29, 2013 at 4:56 pm

    [...] EdEval Week: Shelly Engelman and Tom McKlin on assessing the evolution of Social Networks Using Node… [...]

    Reply

  • Marc Smith on Using NodeXL · AEA365 · March 30, 2012 at 3:42 am

    [...] I’m Marc Smith, Chief Social Scientist at Connected Action, co-founder of the Social Media Research Foundation, and one of the team behind NodeXL I’ve been excited to see posts on aea365 from users of NodeXL such as Johanna Morariu, and Shelly Engleman and Tom McKlin. [...]

    Reply

  • APS · May 10, 2011 at 7:39 pm

    This is a great approach to evaluating changing relationships among organizations. The next step is figuring out how to take advantage of the network effect. Have you thought about which type of educational programs would increase in value the more participants there are?

    Thanks

    Reply

Leave a Reply

<<

>>

Archives

To top