Lisa Garbrecht on Overcoming Data Quality Challenges in School-Based Settings

Hello, my name is Lisa Garbrecht. As a Research Associate at EVALCORP Research & Consulting, I work on numerous projects requiring data collection from youth. As you may know, it is not always easy to obtain high quality data (i.e., sufficient numbers of completed surveys, academic data, etc.) when relying on schools to help facilitate the data collection process. Below are a few tips that have proved useful!

Hot Tip #1:  Take time up front to identify the right data and methods. With the limited time and resources faced by schools and school-based programs today, it is important to collaborate with clients early on to identify priority needs and ensure that data are collected efficiently. Would a brief post-survey suffice instead of a comprehensive pre-post? Be strategic and include only items that really matter. Phrase items clearly and simply to ensure they are easily understood. Show schools you respect their time by only asking for the most vital information to inform the evaluation. 

Hot Tip #2:  Partnerships are key. Working together, evaluators and clients can build mutually beneficial relationships with schools to overcome their resistance to providing data. By showing school personnel and stakeholders how the findings may be of use and providing them with the necessary tools and databases, schools are more willing to collect and provide data in a timely manner. Communicate regularly with clients and schools, providing contact information so that you can answer their questions and offer assistance as needed.

Hot Tip #3:  Look at the data before it is too late. Whenever possible, do not wait until the end of the data collection process to analyze what’s coming in. Running the data early on allows you to identify problems with the tool or data collection process and make changes. Monitor data quality on at least a quarterly basis. This allows you to provide clients and schools with formative information that can serve to strengthen their programs and their motivation for assisting with ongoing data collection.

Hot Tip #4:  A little incentive goes a long way. Use incentives with project staff, school personnel and/or students as allowed. For instance, EVALCORP uses an award system for rewarding site staff members who consistently collect accurate, legible and complete survey data with a small gift card and certificate of appreciation. Pizza parties or other youth-friendly activities are other alternatives for showing clients/schools your thanks. If tangible incentives are not possible, be sure to let those involved know the value of their input and how much you appreciate their time. Oftentimes, a “Thank you and I really appreciate your help” goes a long, long way!

This aea365 Tip-a-Day contribution comes from the American Evaluation Association. If you want to learn more from Lisa, check out the sessions sponsored by the PreK-12 Educational Evaluation TIG on the program for Evaluation 2010, November 10-13 in San Antonio. If you would like to contribute an aea365 Tip, please send an email to aea365@eval.org.