Hi! I’m Elizabeth McGee, founder of LEAP Consulting. LEAP helps nonprofits, foundations, and public institutions achieve impact through meaningful evaluations and strategic support. As a consultant specializing in evaluating systems change, I have the opportunity to work with clients whose efforts target large-scale systems, and this requires the ability to simplify seemingly abstract initiatives in order to design effective evaluations.
In recent decades, “systems thinking” has become more common place in both the public and private sectors and has been proven to be a more effective way to solve complex societal problems. Systems change work is inherently complex and can feel abstract, thus evaluations for these initiatives need to be set up in a way that responds to this complexity by clearly identifying the changes we hope to see at different levels and how those changes build on one another to create the desired population-level impact. At LEAP Consulting, we use framing adapted from ORS Impact & Spark Policy Institutes’ evaluation of Collective Impact Initiatives. This framing helps our clients articulate the changes they want to see as a result of their initiatives’ work. Specifically, it helps to examine the contribution to large-scale impact, as well as progress towards short-, mid- and long-term goals on route to large-scale impact.
This framing, which classifies systems changes at four discrete levels: 1) systems change strategies, 2) early changes in the landscape, 3) longer term systems changes, and 4) population change, has helped our clients think more concretely about how systems change cascades over time, and has also helped us as evaluators of systems change, determine the most appropriate evaluation method(s) for each level of change.
Hot Tip: When evaluating complex systems initiatives, spend time on the front end of the initiative working with the client to develop a Theory of Change (ToC) using these framings above (i.e. systems change strategies, early changes in the landscape, and long-term systems change, and population change). See the ToC as a “living document” that can be revisited and revised on a regular basis to align with the strategic direction of the initiative if it evolves over time.
Lesson Learned: A Theory of Change (ToC) will become important to maintain a clear perspective of how all the initiatives components at different levels work together to contribute to changes within the target population that is benefiting from your systems change. A ToC can also be a helpful tool for framing and contextualizing evaluation findings and data points later in the initiative.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.