Hi, we are Miranda Yates and Anne Gleason from the Program Evaluation and Planning (PEP) Department at Good Shepherd Services in New York City. As part of our agency-wide commitment to youth and family development and evidence-based practice, we recognize the profound value of partnering with youth participants at all stages of program evaluation. Our approach supports youth to identify powerful questions, select evaluation methods, collect and analyze data, interpret findings, and carry out plans to strengthen programs. In particular, we have found that focus groups with youth participants are an important means of clarifying the processes though which programs make a difference.
Hot Tips:
Inspired by all we learned at the 2012 summer session of the Critical Participatory Action Research (CPAR) Institute, we use two particular techniques to engage participants in our program evaluation process.
1. MAP ACTIVITY. We start focus groups by passing out blank paper and colored pens and asking everyone to spend 10 minutes visually representing their experience in a particular Good Shepherd program. We then have all youth in the group share what they depicted. This has proven to be a great way to get everyone engaged in conversation, surface common themes and metaphors, and better understand the program’s impact. Below are two examples of maps drawn by participants in our educational programs which assist over-age and under-credited youth in earning their high school diplomas and successfully transitioning to college. Both maps highlight the difference made by strength-based programming and an emphasis on social support.
2. INTERPRETING SURVEY FINDINGS. Another engaging activity is to present program data to participants and ask for their help interpreting it. PEP administers a number of surveys to participants to assess changes in participant behaviors, attitudes or knowledge. Understanding that we can be limited in our ability to interpret data findings from youth surveys, we recently asked a group of youth to help us interpret results from the Youth Experience Survey. The youth offered critical insights into why certain aspects of the program were rated higher than others. These were conclusions that we would not have come to on our own and has allowed for a much richer analysis of the survey data.
Rad Resource:
For more ideas on approaches to engaging groups in meaningful conversations, check out resources from People Potential.
The American Evaluation Association is celebrating Youth Focused Evaluation (YFE) TIG Week with our colleagues in the YFE Topical Interest Group. The contributions all this week to aea365 come from our YFE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Conducting a program evaluation about a youth program, whilst engaging the young participants in the process is a fantastic idea. So many 21st century learning skills are demonstrated in this process; ones that will transfer perfectly into the 21st century workplace. Their role in the program evaluation, with their insights and new perspectives echoes the work they do in the Good Shepard programs. I particularly like how the two strategies outlined in this article that engage the youngsters in the PE is an extension of the values and goals of the program; ones that will help them succeed in the future. Thanks for sharing!
This is a great tip! Just the other day, I was working with someone who will need to conduct a program evaluation (and my role is to teach evaluation concepts and help develop the plan) and said to me “I never even thought about including the kids in program eval! How would we do that?” I’m going to share these strategies (and others from this week) with her. Thanks!