Hello, we are Dina Wilderson (Director of Learning & Evidence at REDF), Pablo Gaxiola (Director of Reintegration Programs at Goodwill of Silicon Valley), and Jennifer Laird (Senior Researcher at RTI). We’re here today to talk about an evaluation project we collaborated on that included participant feedback as a core component.
Our three organizations, in addition to Fund for Shared Insight, partnered to conduct an evaluation examining the impact of employment social enterprises (ESEs) on economic self-sufficiency and life stability. What exactly is an employment social enterprise? The ESE intervention blends an income generating business with paid employment, job skills training, and wraparound services to bolster the talent of individuals breaking through barriers to employment. An Evaluation Learning Committee was created to shape the evaluation and included representatives from RTI, ESE staff members (including program alumni), REDF program leadership, and Fund for Shared Insight (experts in feedback research).
Lessons Learned
Here are some key takeaways we found from the evaluation findings and process:
Jen: The inclusion of perceptual feedback into the study design yielded great insights, particularly because this was the first time that perceptual feedback findings were combined with an impact study and linked to post-program outcomes. In this component we looked at whether employees’ perceptions of their program are associated with their later employment and life stability outcomes. What we found was the validation that the environment created within an ESE matters and that the relationship with ESE staff is important to the attainment of outcomes.
Dina: The Evaluation Learning Committee (ELC) played such a vital role in the evaluation process, providing direction and guidance at all points. The ESE leaders brought a unique perspective and because of it we had better data collection tools, sharper data collection processes, and the benefit of their insight in the interpretation of results. One specific example that shows the impact of the ELC is in the way that ESE leadership supported the survey process. They helped design strategies to contact participants who were no longer in the program for the follow-up portion of the study and they supported the effort, utilizing outreach strategies that have been effective for programming to support the evaluation study. This resulted in an incredibly strong response rate that could not have been achieved without their guidance. Another thing that has meant a lot to me is that participation in the ELC built evaluation capacity within the enterprises.
Pablo: Being a part of the evaluation project and the ELC has contributed to both my growth and the organization’s growth. We made program changes based on what came out of the perceptual feedback findings, for example, around responses to the statement, “I’m fearful that I won’t be able to succeed in another job outside GWSV.” What we learned was that those who were more optimistic about finding a job post-program were more likely to be successful in later employment, so we wanted to find a way to boost those feelings of self-confidence during the program. We added a programming element where employees come back and meet with current participants. Hearing from living, breathing examples of what can be after leaving is inspiring, even just seeing alumni walking through the hallways in their professional attire, chef hats, and union construction gear is powerful. And we have incorporated participant focus groups into our client feedback process, taking what we learn to drive program changes.
We think the best way to sum things up is we’d do it all over again. And others should try this approach on for size because the benefits are multifold.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.