Hi, this is Barbara Heath and Aruna Lakshmanan of East Main Educational Consulting, LLC. Our group focuses on the evaluation of science, mathematics, and technology programs in K-12 schools and institutes of higher learning.
Most evaluation methods assume that the organization or program being evaluated is stable, controllable, and predictable. When the organization is complex, and constantly changing to adapt to its environment, a systems-based evaluation may be a better choice for the evaluator. A systems perspective provides the evaluation team with a framework to investigate beyond activities and their anticipated outcomes.
Our group has been applying systems tenets to evaluate the development of a multi-institutional, interdisciplinary collaborative. The framework provides the opportunity to see the collaborative as the developing, changing system that it is in reality.
Within the systems framework, our team has selected Eoyang’s CDE Model, which blends tenets from Complex Adaptive Systems (CAS) and Human System Dynamics (HSD). This model represents conditions that exist for self-organizing behavior to occur: containers (C), differences (D), and exchanges (E). It provides the appropriate mechanism to demonstrate and capture the complexities of the collaborative in the differences and exchanges that take place across the boundaries of the containers within the system. Using this method has resulted in a more complete understanding of the collaborative, which in turn, has improved the quality of information that we can provide to the client.
Hot Tip #1: Enter this process knowing that systems analysis takes a great deal of time. It requires good organizational skills, the ability to work with detail while being able to understand the big picture, and time to brainstorm.
Hot Tip #2: Our data analysis parallels the processes used for qualitative analysis: unitizing, categorizing, and linking categories to identify trends are essential steps to the process.
Hot Tip #3: Having graphics support is important. The reporting of data and results from systems analysis is challenging and our results require the extensive use of visualization methods to represent the interactions between the various components of the system and how they influence each other.
Rad Resource #1: Williams, B., & Imam, I. (Eds.). (2007). Systems Concepts in Evaluation. Point Reyes: EdgePress of Inverness.
Rad Resource #2: Systems workshops offered at the AEA Annual Conference.
The American Evaluation Association is celebrating Systems in Evaluation Week with our colleagues in the Systems in Evaluation AEA Topical Interest Group. The contributions all this week to aea365 come from our Systems TIG members and you may wish to consider subscribing to our weekly headlines and resources list where we’ll be highlighting Systems resources. You can also learn more from the Systems TIG via their many sessions at Evaluation 2010 this November in San Antonio.