Hi! We are Krista Collins, Director of Strategy & Innovation at Boys & Girls Clubs of America (BGCA), and Mike Armstrong, Vice President of Club Operations and Evaluation at Boys & Girls Clubs of Metro Atlanta (BGCMA). Together we seek to understand how our professional development courses and youth programs work in tandem to support the 58,000 staff members in local communities and across the nation that create opportunities for approximately 4 million youth each year to achieve great futures through our priority focus on Academic Success, Good Character and Citizenship, and Healthy Lifestyles.
Since 2011, BGCA has conducted the annual National Youth Outcomes Initiative (NYOI) to measure how effectively the Club experience is being implemented and its’ impact on our members. Built on research-informed indicators of youth achievement that align with our priority outcomes, and benchmarked against other leading national youth surveys, NYOI data is used to drive continuous quality improvement efforts and communicate our impact to key stakeholders across the youth development field.
Rad Resource: Looking for comparison data to understand the impact of youth development programs? Download our 2015 National Outcomes Report: Measuring the Impact of Boys & Girls Clubs. A few highlights from our report:
- 74% of members aged 12-17 who attend the Club regularly say they earn mostly A’s and B’s, compared to 67% of youth nationally.
- By 12th Grade, Club members’ rate of monthly volunteering is more than double that of the national average for same-grade peers.
- Teens who stay connected to the Club as they get older seem better able to resist high-risk behaviors than teens nationally at the same ages.
Hot Tip: Sharing Club-level results and training on data-utilization promotes survey participation
In four years the number of NYOI participants has grown from 2,800 Club members to 165,000 – that is an increase of almost 6000%! Much of this growth can be attributed to BGCA’s efforts to demonstrate the value of data to local Clubs. BGCA prepares reports for each participating Club organization, and provides local trainings and consultations to ensure that the results are interpreted correctly and used to drive improvement.
Hot Tip: Data-utilization requires learning that is strategic and intentional.
To fully realize the value that formal measurement and evaluation brings, local clubs have employed continuous quality improvement systems that integrate knowledge generation and decision-making at all levels of their organization. Decision making that affects everything from resource allocation at the corporate level to programmatic foci and staff assignments at the club-site level only occurs if a formal and iterative process of reflection and dialogue practiced.
We’re looking forward to October and the Evaluation 2016 annual conference all this week with our colleagues in the Local Arrangements Working Group (LAWG). Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to contribute to aea365? Review the contribution guidelines and send your draft post to aea365@eval.org.
Dear Krista and Mike,
I appreciate you sharing your thoughts on youth program evaluation within the Boys & Girls Clubs of America. I am currently a Grad student who is particularly interested in program evaluation within the education sector and youth. I had the chance to read the most updated 2016 National Outcomes report for the impact of BGCA, and I found the report particularly interesting with regards to the outcome driven experience. With this of 5 element framework of outcome driven approach, I found the evaluative theory easy to understand the different indicators. I also appreciate that the 5 elements are hierarchical in nature from the first element focused providing a safe and positive environment, to last one providing youth with validation and recognition.
The simple approach of having specific experience components linked to outcomes allowed for a better way to identify areas of improvement for club experience. Another interesting feature of this report was how the data was measured to differentiate outcomes based on 3 classes of youth (youth, younger teens, and older teens). This has given me greater insight on how I can also consider the variability in age groups within my program evaluation.
Lastly, I was really inspired to see how the participants in your evaluative program increased 6000% in just over 4 years, and its quite telling that you attribute this success to sharing the data with stakeholders and program implementers. In your efforts to push a culture that is focused on making data driven improvements, I’m very interested in knowing what strategies worked well with regards to making and sharing reports? Did you hire specific a PR firm to disseminate this information? What aspects of report making did you find particularly useful in inspiring clubs to easily interpret data, and drive improvement?
Thank you for sharing your success in this post and inspiring others to also conduct meaningful evaluations which are very well organized.
Regards,
Ayoun Basharat
MEd., Assessment and Evaluation
Secondary School Chemistry Educator