We are Tanushree Banerjee, doctoral student in the Research, Assessment and Evaluation concentration and Dr. Lisa Abrams, a Professor of Research, Assessment, and Evaluation in the School of Education at Virginia Commonwealth University. We are part of a research team working to evaluate the implementation and outcomes of a teacher residency program (TRP).
In Virginia, similar to other states in the US, there are multiple routes to teacher preparation and licensure, including traditional, alternative and residency models. Over the past decade, residency models have become more widespread as they provide a full-year of pre-service training in a K-12 classroom alongside an in-service mentor teacher. Residency programs have core components – undergraduate or graduate coursework, the residency – including a year-long clinical placement in a classroom, mentoring provided by an in-service teacher and a university-school district partnership to support the program. In this blog post, we will highlight how we are applying systems-thinking ways to evaluate educational programs, like a residency model of teacher preparation.
Systems-thinking approaches allowed us to use exploratory process-tracing to map the pathway of the components, activities, outputs and the association and interactions of the components to examine program effectiveness and interpreting program outcomes. Below is a summary of a few lessons we learned when applying systems-thinking approaches to a pre-service residency preparation program.
Lesson Learned:
Generally, residency models have four core components including a K-12 school districts-university partnership, high quality mentor teachers, undergraduate or graduate curriculum, and a year-long clinical experience. The systems diagram below shows how pathway mapping / influence mapping these components demonstrates the interconnectedness and non-linear relationships. Depicting the program components and inter-relationships in this way illustrates possible causal mechanisms that contribute to program outcomes in ways not necessarily visible in traditional, more linear, logic-model program theory. This depiction has helped us to identify key connections and interactions among more refined program components that can lead to the desired TRP outcomes.
Hot Tip:
The logic model most often provides a linear perspective of a system representing a program theory that fails to capture relationships between the program components. Embracing systems-thinking approaches can advance our understanding of how and why a program works or is not working as intended and accounts for the complexity of programs with multiple dimensions, participant groups, stakeholders and program activities. Systems-thinking approaches can provide a framework for evaluation practitioners in process and outcome evaluations to address nonlinearity, complexity and context of the program and will enhance the evaluation use and evaluation findings.
Rad Resource:
- System Diagrams: A Practical Guide. By Bob Williams (Wellington, Aotearoa New Zealand: 2021)
- Systems Concepts in Evaluation: An Expert Anthology, Edited by Bob Williams & Iraj Imam (Point Reyes, CA: Edge Press, 2007)
The American Evaluation Association is hosting this week with our colleagues in the Systems in Evaluation Topical Interest Group. The contributions all this week to AEA365 come from SETIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.