Ed Eval TIG Week: Next Steps for Continuous Quality Improvement by Krista Collins

Hi, my name is Krista Collins, Director of Evaluation at Boys & Girls Clubs of America (BGCA) in Atlanta, GA.  Over the past few years, after school program quality standards have become more prevalent across the field as a way to ensure that young people are engaging in safe and supportive environments that promote positive developmental outcomes.  The design and implementation of Continuous Quality Improvement (CQI) processes has therefore increased rapidly as a methodology to monitor and improve program quality.  While all grounded in a similar feedback loop of design, test and revise, the models below are a few common examples of the various CQI frameworks that are being used within and across sectors.

In 2012, The David P. Weikart Center for Youth Program Quality released the results of an empirical study to test the impact of their continuous improvement process, the Youth Program Quality Intervention (YPQI) on program quality in after school systems. Their findings showed that YPQI had a significant positive impact on youth development practice and staff engagement, with outcomes sustained over time across multiple after school contexts. Within K-12 schools, quality improvement processes are often foundational to school reform efforts to turn around consistently low-performing schools. Studies have shown that when school reform includes a commitment to a specific strategy or plan (design), assessment of teacher and student performance (test), and opportunities for learning and improvement (revise), then positive impacts on teacher preparation, instruction, and student achievement are more likely (Hargreaves, Lieberman, Fullan & Hopkins, 2014; Hawley, 2006).

Lessons Learned: While CQI has garnered widespread support across industries, efforts to monitor and evaluate its effect have been limited due to challenges associated with the highly contextualized and iterative nature of CQI.  In a report from the Robert Wood Johnson Foundation, they summarized that the continuous evolution of design, metrics, and goals makes it difficult to determine if actual improvement has been made, and the learnings gained have limited generalizability.  These challenges, coupled with the long timeline required, have motivated new quality improvement methods to be identified.

Hot Tip: Generated in the healthcare space, The Institute for Healthcare Improvement has developed the Breakthrough Series Collaborative (BCS), an innovative approach to CQI that prioritizes the need for and value of rapid improvement with an emphasis on the team structure and procedures needed for efficient implementation. Their own healthcare evaluations, as well as studies examining the impact of this methodology to improve Timely Reunification within Foster Care, have shown significant and timely improvements in service delivery, stakeholder engagement and outcomes, cross-system collaboration, and reduced costs.  These successes demonstrating the value of BCS as a methodology to improve current CQI models warrant consideration and testing with the PreK-12 Education and after school space.  With the ever-increasing need to ensure that young people are exposed to the high-quality learning environments required to drive positive outcomes, the advantages of BCS may provide a more efficient and robust solution to drive effective school reform and quality improvement efforts.

The American Evaluation Association is celebrating Ed Eval TIG Week with our colleagues in the PreK-12 Educational Evaluation Topical Interest Group. The contributions all this week to aea365 come from our Ed Eval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.