This is Jean King and Gayra Ostegaard Eliou, from the University of Minnesota, members of the Complex Adaptive Systems as a Model for Network Evaluations (CASNET) research team. NSF funded CASNET to provide insights on (1) the implications of complexity theory for designing evaluation systems that “promote widespread and systemic use of evaluation within a network” and (2) complex system conditions that foster or impede evaluation capacity building (ECB) within a network. The complex adaptive system (CAS) in our study is the Nanoscale Informal Science Education Network (NISE Net), a network that has been continuously operating for ten years and is currently comprised of over 400 science museum and university partners (https://player.vimeo.com/video/111442084). The research team involves people from University of Minnesota, the Museum of Science in Boston, the Science Museum of Minnesota, and the Oregon Museum of Science and Industry.
This week CASNET team members will highlight what we’re learning about ECB in a network using systems and complexity theory concepts. Here is a quick summary of three lessons we learned about ECB in a network and systems readings we found helpful.
Lessons Learned:
- ECB involves creating and sustaining infrastructure for specific components of the evaluation process (e.g., framing questions, designing studies, using results). Applying a systems lens to the network we studied demonstrated how two contrasting elements supported ECB:
- “Internal diversity” among staff’s evaluation skills (including formally trained evaluators, novices, thoughtful users, and experts in different subject areas) provided a variety of perspectives to build upon.
- “Internal redundancy” of skill sets helped ensure that when people left positions, evaluation didn’t leave with them because someone else was able to continue evaluative tasks.
- ECB necessitates a process that engages people in actively learning evaluation, typically through training (purposeful socialization), coaching, and/or peer learning. The systems concepts of neighbor interactions and massive entanglement pointed to how learning occurred in the network. NISE Net members typically took part in multiple projects, interacting with many individuals in different roles at different times. Network mapping visually documented the “entanglement” of people from multiple museums, work groups, and in numerous roles that supported ECB over time.
- The degree of decision-making autonomy a team possessed influenced the ways in which–and the extent to which–ECB took place. Decentralized or distributed control, where individuals could adapt an evaluation process to fit their context, helped cultivate an ECB-friendly internal organizational context. Not surprisingly, centralized control of the evaluation process was less conducive to building evaluation capacity.
Rad Resources:
- Davis and Sumara’s Complexity and education: Inquiries into learning, teaching, and research
- Williams and Hummelbrunner’s Systems concepts in action: A practitioner’s toolkit
- Eoyang and Holladay’s Adaptive action: Leveraging uncertainty in your organization
The American Evaluation Association is celebrating Complex Adaptive Systems as a Model for Network Evaluations (CASNET) week. The contributions all this week to aea365 come from members of the CASNET research team. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.