Hello! We are Karen Peterman (Karen Peterman Consulting, Co.) and Marah Moore (Director of i2i institute and ECLIPS coPI). We are going to talk about some of the implications of a systems approach for the role of the evaluator.
In a recent Thought Leaders Forum, Michael Patton reminded us that evaluation is a transdisciplinary field. Evaluators need expertise in evaluation theory, practice, methods and use, as well knowledge of theories of change and how to generalize what they have learned about patterns in effective interventions. A systems approach to evaluation can enhance evaluators’ work across each of these areas of expertise.
I (Karen) found the ECLIPS’ biggest impact on me was in how I view my overall role as an evaluator. And I (Marah) found that the evaluator’s role was revisited throughout the process of addressing each new systems concept. Here are some ideas we came away with:
Lesson Learned: Use your “systems-based evaluation expertise” to add value for clients.
Systems concepts can provide a valuable lens through which to view project evaluation findings. For example, you can help clients consider their individual projects within the context of academic literature, their institution’s larger mission, and/or their funders’ goals.
To guide these discussions, check out ZIPPER, A System-Based Evaluation Mnemonic and the Systems Archetypes.
Lesson Learned: Use a systems approach to push evaluation beyond the traditional stages.
The ECLIPS graphic below shows a traditional and a systems view of the stages of evaluation.
The systems view encourages the evaluator to work at the intersection of the traditional stages. The evaluator asks clients and participants to help shape:
- the evolution of the evaluation plan,
- the data collection procedures, and
- the interpretation of results.
Bringing together a systems orientation and participatory evaluation leads to an evolution in evaluation practice.
Hot Tip: Break down the silo approach to evaluation.
At AEA last fall, a number of presenters stated that the purpose of their work was to improve education. The systems perspective provides methods to start thinking about and achieving that goal on a broader scale. It enhances the thinking and work of evaluators by providing a framework and tools to move beyond the evaluation of the immediate project to start challenging clients (gently!) to think about system-level change and how their projects really can make a big-picture difference.
Read the final ECLIPS blog tomorrow where Tarek and Matt give an overview of systems- oriented evaluation and look at some fuzzy logic models.
Rad Resources:
- Karen’s presentation at the 2012 AEA conference:Blurring the Lines: The Process of Applying Systems Thinking to an Ongoing Evaluation.
- ECLIPS on the InSites website
The American Evaluation Association is celebrating this week with our colleagues involved in ECLIPS—Evaluation Communities of Learning, Inquiry, and Practice about Systems—and the AEA Systems in Evaluation Topical Interest Group. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Great post! So what is the shape of practice? If you can’t see it, use evaluation to push the systems approach beyond the traditional stages.
Chad