Systems- and Complexity-informed Evaluation Week: Five Tips for Evaluation of System Change by Emily Gates and Francisca Fils-Aime

I’m Emily Gates. I teach, write about, and practice evaluation at Boston College. My work focuses on the intersections of values, systems, and equity in evaluation, particularly in public health and education. I’m Francisca Fils-Aime, a PhD student at Loyola University Chicago. My research interests include international systems and global peace.

We studied an exemplar case of a system change initiative in public health. Given the growing interest in system change, we thought we’d share a few tips. First, a bit of background on the case study. We partnered with the Rippel Foundation which works to change the systems and conditions that influence health and wellbeing through its ReThink Health initiative. In coordination with Bobby Milstein, Director of System Strategy, and Jane Erickson, Director of Evaluation & Learning, we sought to understand how ReThink Health envisions, enacts, and evaluates its efforts to change systems through group and individual staff interviews and document review. Here are a few tips from what we learned.

Hot Tip #1

Systems change requires reframing problems and cultivating stewardship. Changing systems requires reexamining the root causes of intersecting social, educational, health, and environmental problems and the boundary assumptions that constrain potential responses. Two meaningful shifts for ReThink Health were reframing focuses on healthcare to health and wellbeing and assumptions of health as separate to seeing its interconnections with education, employment, housing, etc. Systems change also involves helping people across roles and institutions grow their sense of responsibility and power to influence change.

Hot Tip #2

Build evaluation to fuel learning first, accountability second. Anticipate a tension between learning and accountability as driving purposes for evaluation and weigh the benefits and limits of each. Embed norms and outcomes of learning within the system change process and at multiple scales. Keep in mind that some stakeholders will still want to know what was done and achieved so embed accountability feedback in meaningful ways. 

Hot Tip #3

Use adapted and new tools. Explore the multitude of systems approaches available for problem structuring, system analysis, group learning, narrative change, etc. Creatively adapt approaches and tools to suit the purposes, uses, and contexts. For example, ReThink Health uses an adapted theory of system change called Stewards’ Pathway, customizes system dynamics modeling, develops essential stewardship practices, among other resources.

Hot Tip #4

Assess contributions to changes in system conditions. Shift from a traditional focus on changes in individual and population-level knowledge, attitudes, and behaviors to assess systemic conditions. Look for how initiatives are contributing to shifts within these conditions and let go of linear cause-effect relationships and attribution. While studying these shifts are just beginning in the field, some ideas for what to track come from research on complex adaptive systems and network science.

Hot Tip #5

Shift the role of evaluation (and evaluator) from judging to codeveloping value. Reshape evaluation from rendering discrete assessments of performance to facilitating ongoing evaluative processes and deliberation among those involved and affected about the value of what they’re up to and what should be done next. Systems change is a long game, in which conflicting perspectives and shifting evidence are likely. Evaluative processes should provide ways to continuously codevelop value not only answering questions of what worked, for whom, in what circumstances but also what should be done next amidst change. See the full argument for this shift in a new book by Thomas Schwandt and Emily Gates.

The American Evaluation Association is hosting Systems- and Complexity-informed Evaluation Week. The contributions to AEA365 this week are all related to this theme. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.