Hello, AEA365 community! Liz DiLuzio here, Lead Curator of the blog. This week is Individuals Week, which means we take a break from our themed weeks and spotlight the Hot Tips, Cool Tricks, Rad Resources and Lessons Learned from any evaluator interested in sharing. Would you like to contribute to future individuals weeks? Email me at AEA365@eval.org with an idea or a draft and we will make it happen.
Hi, I’m Kim Norris, Monitoring, Evaluation and Learning (MEL) Director for American Institutes for Research (AIR)’s International Development Division. As co-chair for the Systems in Evaluation Topical Interest Group (SETIG), I get excited about using systems thinking in evaluation work to improve evaluations. In this case, I am reminded of how systems thinking in evaluation (STE) helps to move us toward localization.
Briefly, localization involves adapting efforts to meet the needs and preferences of a specific context or culture. It is about shifting focus to local actors—valuing their knowledge, respecting their expertise, championing their agency, and engaging them as partners and leaders rather than as agents and beneficiaries. Systems thinking in evaluation (STE) is a mindset that helps us to understand the complexity and interdependence of social and ecological systems we work in. STE involves applying core systems concepts: perspectives, boundaries, interrelationships, and dynamics, to inform evaluation designs, methods and use.
By applying STE to our evaluation practice, we can support localization in several ways: enhance contextual understanding and cultural sensitivity; strengthen stakeholder engagement and collaboration; improve evaluation quality and relevance; and increase evaluation use and impact.
Perspectives: STE invites us to engage with multiple and diverse viewpoints of system stakeholders, including those who are marginalized or excluded to better understand the values, assumptions, interests, and power dynamics that shape the system behavior and outcomes. Applying STE can foster dialogue and learning among different groups and promote participatory and inclusive evaluation processes that respect and empower local actors.
Boundaries: STE challenges us to reflect on the choices we make about what to include or exclude in our evaluation scope, criteria, methods, and reporting. In doing so, we can better justify our decisions and acknowledge their implications for the validity, reliability, credibility, and usefulness of our evaluation processes and findings. Co-creating or validating boundary judgments increases transparency and accountability to our stakeholders.
Interrelationships: STE encourages us to identify and analyze connections and feedback loops among the elements of a system, such as actors, activities, resources, outcomes, and context. Through STE we can better appreciate the diversity and interdependence of local actors and their roles in achieving systemic change. We can also recognize the potential synergies and trade-offs among different interventions and outcomes and avoid unintended consequences or negative externalities.
Dynamics: STE enables us to observe and explain the patterns and trends that emerge over time in a system, such as growth, decline, oscillation, or chaos. These often hinge on an ongoing understanding of local contexts and actor considerations and enable us to better anticipate and respond to changes and uncertainties affecting the interventions and outcomes we are evaluating. We can better monitor and evaluate any systemic effects of program or evaluation actions, and support strategy adaptations.
STE is not a prescriptive approach or specific method. It’s a mindset that can be applied to any evaluation context or purpose. STE has its own challenges. It can be time-consuming and resource-intensive, be difficult to communicate or report, be subject to bias or manipulation, and raise ethical or political issues. We need to be critical and reflective when applying STE to evaluation practice, be aware of the assumptions and risks involved in STE and seek feedback and learning from others using STE.
Rad Resources
Peter Stroh’s Systems Thinking for Social Change: A Practical Guide to Solving Complex Problems, Avoiding Unintended Consequences, and Achieving Lasting Results.
Another RAD Resource: Urban Institute: Nine Ways to Strengthen Program Evaluations by Centering Community Voice | Urban Institute.
For more, get involved with the Systems Evaluation Topical Interest Group.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.