CAP Week: Mehmet Dali Ozturk on Evaluation of Summer Outreach Programs

Greetings, My name is Mehmet Dali Ozturk. I am the Assistant Vice President of Research, Evaluation and Development at Arizona State University Office of Education Partnerships (VPEP), an office that works with P-20, public and private sector partners to enhance the academic performance of students in high need communities.

Along with my colleagues Brian Garbarini and Kerry Lawton, I have been working to develop sound and reliable evaluations to assess educational partnerships and their ability to promote systemic change. One of my ongoing projects has been the evaluation of ASYouth, a program developed to provide a holistic support system to the University, schools, and parents so that disadvantaged children have the opportunity to participate in university-based summer enrichment activities.

Based on this experience, we offer the following advice to evaluators working on university-based outreach programs:

Hot Tip: Create a Multi-Disciplinary Evaluation Team

Although most University-led summer enrichment programs are directed towards similar goals, the activities often focus on a multitude of subjects ranging from drama, music and art to intensive math and science courses. Given this, evaluation teams that recruit individuals with expertise in a variety of academic subjects are well-equipped to develop evaluation designs and assessment tools appropriate to these programs.

Hot Tip: Ensure Linguistic and Cultural Relevance

Evaluations should be developed and conducted by evaluation teams that possess cultural competency to the target population. This allows for the development of culturally sensitive assessment materials that can be translated into the heritage language of the program participants at a fraction of the cost of hiring outside consultants. In addition, when survey methods are used, culturally-appropriate measures will result in higher initial response rates. The need for fewer follow-ups can greatly reduce the cost of successful evaluations.

Hot Tip: Embed Evaluation into Program Design

Due to limited resources, evaluation expertise, and/or capacity, many summer enrichment programs do not include rigorous evaluation components. In these cases, evaluation is merely an afterthought, making it very difficult to ensure valid data collection or implement a design with appropriate controls.

This aea365 contribution is part of College Access Programs week sponsored by AEA’s College Access Programs Topical Interest Group. Be sure to subscribe to AEA’s Headlines and Resources weekly update in order to tap into great CAP resources, and to consider attending CAP-sponsored sessions this November at Evaluation 2010.