Hello, AEA365 community! Liz DiLuzio here, Lead Curator of the blog. This week is Individuals Week, which means we take a break from our themed weeks and spotlight the Hot Tips, Cool Tricks, Rad Resources and Lessons Learned from any evaluator interested in sharing. Would you like to contribute to future individuals weeks? Email me at AEA365@eval.org with an idea or a draft and we will make it happen.
Howdy AEA 365, it’s Will Fisher and Jenny Seelig, Research Scientists with NORC at the University of Chicago. NORC is devoted to objective and dynamic social science research.
As originally planned, our study, Engaging Youth for Positive Change (EYPC): Promoting Community Health Through Civic Education, was a randomized controlled trial carefully designed to evaluate the impact the EYPC’s[i] civics curriculum has on student health and community well-being in rural Illinois. It was funded by the Robert Wood Johnson Foundation in 2019 and scheduled to take place from 2020 to 2023. By Spring 2020, we had recruited 18 schools and 18 teachers into control and treatment groups and expected to steadfastly proceed. However, no one could have predicted the circuitous path our research would take.
From 2020-2023, school systems faced the triple challenge of mitigating the COVID pandemic and social upheaval in the wake of George Floyd’s murder, and jaw-dropping anti-government action., Their attention stretched to supporting their students, families, and communities as they mitigated associated trauma – in addition to educating students. And rightfully so. However, our evaluation was sidelined as a result, forcing us to reassess our research efforts How could we realign our work to understand how a civics curriculum, student health, and community well-being interact and influence each other, given the new landscape?
How we developed and Implemented a Plan B
When most schools across the country decided to move towards online and hybrid learning in Fall 2020, only two of our study original participants remained on board. Therefore, we expanded recruiting to rural schools from other states. Throughout Summer and Fall 2021, the research team called over 1,200 principals and e-mailed social studies teachers and their district administrators, with the invitation to participate in the study. As the re-imagined recruitment process was nearing its conclusion, we realized that we were still far from our school participation goal. Due to this lackluster outcome, we developed and implemented the addition of qualitative data in the form of teacher interviews. Studies that center rural schools as significant sites of research opportunity and program evaluation offer an opportunity to attend to what place-specific research can do best – illuminate the contextual elements of education and how these characteristics both shape and are shaped by what occurs within rural schools.
What We Learned in the face of adversity
The course of the world altered during the 3.5 years of our study and challenging us to change. We learned two important lessons as a result. First, preparing for all contingencies is impossible, and our multiple design pivots illustrate the importance of being open and receptive to changing the research plan. Instead of taking the challenges of recruitment at face value, for example, we opted to adapt and reflect on other effective options. Second, be open to unexpected findings. For example, although the data we collected did not reflect changes in the health or political behaviors of rural youth, our findings suggest that in some locations the EYPC curriculum spurred community change in meaningful and sustainable directions.
[i] Engaging Youth For Positive Change – Meeting Learning Standards Through Civic Engagement (eypcprogram.org)
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.