Greetings! We are Beverly Parsons (InSites and ECLIPS principal investigator) and Veronica Thomas (Howard University and ECLIPS external advisor). Today we introduce you to an exploratory research project, Evaluation Communities of Learning, Inquiry, and Practice about Systems (ECLIPS). It’s funded through a National Science Foundation grant to InSites.
The ECLIPS Community of Practice is 15 evaluators involved in STEM (science, technology, engineering, and math) education evaluations. During 18 months of webinars and annual meetings, ECLIPS members discussed and applied system concepts—especially systems dynamics and complex adaptive systems concepts—to their work. This week we share examples of our learning through this exploratory project.
Lessons Learned: A Systems-Oriented Evaluation?
We’re using a definition of a system from Meadows, Thinking in Systems: “A system is an interconnected set of elements that is coherently organized in a way that achieves something.”What the system achieves may or may not be what we want.
ECLIPS members identified the systems of relevance to their work. They looked at patterns within the systems, paying particular attention to culture and social justice – two areas that I (Veronica) kept in the forefront of the ECLIPS work.
In this type of systems-oriented evaluation, we pay attention not only to activities and results but also to patterns; norms, infrastructure, and policies; and paradigms. You’ll hear more this week about how ECLIPS members have used the iceberg diagram to go deeper in their thinking about systems.
Hot Tip:
Use a systems lens in your evaluation practice to:
- ask different kinds of evaluative questions, including questions that address social justice concerns (e.g., questions about access and opportunity)
- look for patterns that give clues about appropriate theories of change
- find leverage points (where small changes can create large effects),
- consider different roles as an evaluator (e.g., to include social change agent)
This Week with ECLIPS
Tuesday: Dave describes a systems view of a project in the form of a “road map” that shows where interconnections create value.
Wednesday: Lloyd and Pat provide an example of seeing and understanding patterns in systems.
Thursday: David, Ginger, and Alyssa discuss working with boundaries, relationships, and perspectives as leverage points for change.
Friday: Karen and Marah address how a systems orientation influences an evaluator’s role.
Saturday: Tarek and Matt give a wrap-up of systems concepts and connections to fuzzy logic models.
At another time, we’ll share our learning about how to use the systems concepts in more powerful ways than we could do in this exploratory project.
Rad Resources
- ZIPPER, A System-Based Evaluation Mnemonic
- Using Complexity Science Concepts in Evaluation
- Designing initiative evaluation: A systems-oriented framework
- Thomas, V.G. (2011). “Cultural issues in evaluation,” American Journal of Evaluation, 34 (4), 578-582.
The American Evaluation Association is celebrating this week with our colleagues involved in ECLIPS—Evaluation Communities of Learning, Inquiry, and Practice about Systems—and the AEA Systems in Evaluation Topical Interest Group. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.