Hi, I’m Dan Swanson, Assistant Survey Research Manager at Wilder Research. I oversee our survey center where I am responsible for implementing and maintaining quality control measures throughout the data collection process. A phrase often repeated here at Wilder is, “No matter how you analyze or write about the data, its quality cannot get any better once it has been collected.” I’m charged with making sure all data collected at Wilder is high-quality and useful
Hot Tips: Pre-test your survey
No matter how confident you are in the survey or data collection tool you have designed, it is critical that you pre-test it with a small sample first. Until you start reading the questions live and/or get feedback from someone who has never heard or seen the questions before you can never be sure that your instrument is going to work the way you want it to. This is especially important on self-administered questionnaires where you do not have your own trained staff to clarify questions the respondent may not understand.
Hot Tip: Train your data collection staff
Take time before you start a project to make sure your data collection staff understand the protocols and the purpose of the project. Data collection staff are the front line in the research process. The extra time you spend before a project starts informing your staff on why they are collecting the data and who they are collecting it for may just help them convince a few respondents who are on the fence to complete your survey.
Hot Tip: Monitor work in progress
Follow up with your data collection staff early and often. Here at Wilder we conduct live monitoring (audio and visual) on all of our telephone survey projects. We provide immediate feedback to staff on how they introduced the study, avoided refusals, probed for detail, and remained neutral. When feedback is provided in the appropriate manner your staff will see it as an effective tool for improving their ability to collect high quality data.
Lessons Learned: Track data quality
Monitoring and providing feedback is only the first step in the process. We’ve learned how important it is to track information in a database so we can see patterns and problems with individual staff, our staff as a whole, or on particular projects. In addition to tracking this information in a database, we meet weekly to discuss problems uncovered in the monitoring process and inform each other of situations to watch for.
Rad resources:
The American Evaluation Association is celebrating with our colleagues from Wilder Research this week. Wilder is a leading research and evaluation firm based in St. Paul, MN, a twin city for AEA’s Annual Conference, Evaluation 2012. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.