Lija Greenseid on Conducting Mixed-Mode Surveys

I’m Lija Greenseid, a senior evaluator with Professional Data Analysts, Inc. in Minneapolis, Minnesota.

A mixed-mode survey is a survey that collects data by more than one mode, such as telephone, mail, or web. PDA has conducted many mixed-mode surveys as components of our evaluations of state stop-smoking programs, such as tobacco quitlines and in-person stop-smoking programs. Let me share a little of what we’ve learned.

Hot tip: There are many different ways of mixing survey modes. One tried and true method is to mail a printed pre-notification letter in advance of contacting survey participants by telephone or email. Another commonly used design is to contact survey participants by one mode and then follow-up with non-respondents by a second mode. PDA frequently employs a design in which we contact participants by phone up to 7 times, suspend calling to send a mail survey, then continue to phone participants up to 8 more times if they do not respond to the mail survey in a week.

Lesson learned: Mixed-mode surveys can be hard to administer! A survey designer must carefully consider the timing of different modes of contact and how to coordinate the contacts so as not to overburden survey participants with multiple contacts by different modes. This can be automated or done manually with tracking spreadsheets and processes. PDA designed a custom survey system to manage the mixed-mode design described above.

Hot tip: If mixed-mode surveys are harder to administer, why bother? Mixed-mode surveys have the potential of increasing survey response rates, therefore improving survey estimates by reducing non-response and coverage error. Additionally, adding a second lower-cost mode to a phone survey can actually save money by shortening the fielding time and reducing the number of calls. In sum, mixed-mode surveys can get better data for your clients more quickly and at a lower cost. Not too bad!

Rad resource: The best place to start to learn more about mixed-mode surveys is the textbook “Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method, 3^rd Edition” by Don Dillman, Jolene Smyth, and Leah Christian (2009).

Rad resource: The annual conference of the American Association of Public Opinion Researchers (AAPOR) is where cutting-edge research on mixed-mode surveys is presented long before the studies are published in journals. The AAPOR 2012 Conference will be held in Orlando, Florida from May 17-20^th.

Hot tip: I’ll be discussing mixed-mode surveys in an AEA coffee break webinar on Thursday, April 26^th. AEA members, please join me to learn more!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.