CASNET Week: Frances Lawrenz, Amy Grack Nelson, and Marjorie Bequette on Coding Framework for Studying Complex Adaptive Systems

We are Frances Lawrenz and Amy Grack Nelson, University of Minnesota, and Marjorie Bequette, Science Museum of Minnesota (where Amy works, too). We are members of the Complex Adaptive Systems as a Model for Network Evaluations (CASNET) research team. When we started this project, complexity theory seemed exciting, but daunting. What is complexity theory, you ask? Complexity theory, long used by biologists, ecologists, computer scientists, and physicists, has recently been rethought of as a method for facilitating organizational and educational change. Davis and Sumara (2006) suggest that complexity theory can be used as a framework for understanding the conditions through which change can emerge, specifically stating that “complexity thinking has evolved into a pragmatics of transformation—that is, a framework that offers explicit advice on how to work with, occasion, and affect complexity unities” (p. 130).

To wrap our brains around complexity theory, we dug into the literature to understand characteristics of complex adaptive systems (CAS), with a focus on educational networks. Our literature review identified three broad categories of attributes: (1) those related to behaviors within a CAS, (2) those related to agent structure within the system, and (3) those related to the overall network structure.

We wanted to know if the network we were studying was, indeed, a complex adaptive system and, if so, how characteristics of a CAS affected evaluation capacity building within the system. This meant we needed to code our data from a complexity theory lens. We developed a coding framework based both on our extensive literature review and characteristics of complex adaptive systems that emerged from our data. Our coding framework for complex adaptive systems ended up being organized into the following broad categories:

1. Interactions between agents within and outside of the system
2. Decision-making practices within the system
3. Structures within the system to do the work
4. Aspects of system stability
5. Characteristics of the agents
6. Other codes as needed for the specific project

Rad Resources:

We found our literature review matrix and coding framework to be extremely helpful at breaking the concepts into chunks that could be identified in what people did on a day-to-day basis. We’re excited to share our tools here as we think they could be useful to anyone interested in studying evaluation within complex adaptive systems.

• Matrix of the findings from our literature review of complex adaptive system (umn.edu/site)
• Our coding framework for complex adaptive systems in educational networks (umn.edu/theothersite)

Rad Resource:

• Davis and Sumara’s Complexity and education: Inquiries into learning, teaching, and research.

The American Evaluation Association is celebrating Complex Adaptive Systems as a Model for Network Evaluations (CASNET) week. The contributions all this week to aea365 come from members of the CASNET research team. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.