Hello, AEA365 community! Liz DiLuzio here, Lead Curator of the blog. This week is Individuals Week, which means we take a break from our themed weeks and spotlight the Hot Tips, Cool Tricks, Rad Resources and Lessons Learned from any evaluator interested in sharing. Would you like to contribute to future individuals weeks? Email me at AEA365@eval.org with an idea or a draft and we will make it happen.
Hi! I’m Stephanie Mui, Assistant Director of Program Evaluation and Planning at Good Shepherd Services. For the past seventeen years, I have been the internal evaluation for their afterschool programs.
Program evaluation is an essential part of providing quality afterschool programming to youth and their families. Evaluation becomes more powerful and impactful when all of the stakeholders are involved in the evaluation process (O’Donoghue, Kirschner, & McLaughlin, 2003). But how do organizations providing youth programs involve young people in program planning and evaluation?
Youth are the experts on their lived experiences – therefore they are also experts on the supports they need to thrive. For example, since the pandemic, programs have been struggling with meeting enrollment targets and rates of participation are plummeting. Families have indicated that they need afterschool programs to support the family and to help the youth thrive. Yet, these programs have been struggling to identify how best to help families and youth. One of the missing components is that we have not been consistently asking families what barriers exist that prevent their children from coming to the program. We have also not been asking the youth themselves what they would like or what they need from the program as things return to our new normal.
I’ve spent the past several months reflecting on how one agency has been utilizing youth councils in their afterschool programs to include youth voice in program planning. I conducted focus groups with line staff, program directors and agency leadership to identify what factors have supported youth councils in driving involvement of youth in the evaluation process and what barriers exist that impede the process. What lessons can we learn? What are the resources and supports needed in an agency to support youth involvement in evaluation of afterschool programs? To do this, I will be conducting focus groups with afterschool staff and leadership to reflect on how youth voice has been utilized in the past, how programs would like to utilize youth voice in the future and what supports they need to do that.
Lesson Learned: Intentionality
The biggest identified barrier was the inconsistent nature of youth councils across the afterschool programs in the agency. Program leadership identified that in order to effectively support youth councils in the evaluation process, there needs to be intentional planning provided for the youth councils. This includes making sure that the youth councils are included in the program schedule and have identified staff for consistency. Line staff further identified that there needs to be agenda planning and lesson planning done for the facilitation of the youth councils.
Lesson Learned: Technical Supports
Both line staff and program leadership identified that there is a need for technical support in the youth councils. This support includes utilizing the program evaluation department to help with understanding how to teach data skills to the staff and the youth participants. It is challenging for staff to teach evaluation skills to youth as they identified feeling uncomfortable about evaluation themselves.
Recommendations for Moving Forward:
In thinking through the organizational supports that are needed to sustain youth voice in program evaluation, there seems to be a need for both top-down approaches and bottom-up approaches. A top-down approach is one which is driven by agency and program leadership, whereas a bottom-up approach is driven by line staff and youth. Both type of approaches could be beneficial to doing this work. I believe that it is important to have both so that there is investment in youth voice across the agency. If there are only top-down approaches, then it might be difficult to have staff buy-in to the process – it might feel like just more work for already overextended workers. With only bottom-up approaches, program leadership might not see it as a priority, and it becomes one of the first things to go when time/staff gets limited.
Top-Down Approaches:
- Organization having expectation to do program evaluation and to include stakeholder voice in that process – this could include having participation in eval activities in job descriptions
- Training for staff and leadership around utilizing data in program decision making
- Providing resources to support including youth in the evaluation process. This could include stipends for staff to support the process or providing internship opportunities for college students to help support.
Bottom-Up Approaches
- Identifying people who can champion and help implement on the program level, instead of initiative being driven by leadership. These people then can help train their coworkers to become involved in the evaluation process.
- Providing various different opportunities for youth to provide voice on program planning. Some youth may want to participate in the youth councils, while others may feel more comfortable just being in a focus group or completing a survey. It is important to meet the youth where they are at
One possible start to involving youth in program evaluation is to pilot it on a small scale with one or two programs. This will provide opportunities to explore what would work best for the agency, the individual programs, and the youth that they serve. This would then be followed with further reflection on the process so that we can begin to codify a model of youth participation in afterschool program evaluation.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.