Jan Losby and Anne Wetmore on whether to survey or not to survey—that is the question

We are Jan Losby (a CDC employee) and Anne Wetmore (an ORISE Fellow), members of the Evaluation and Program Effectiveness Team in the Division for Heart Disease and Stroke Prevention at the Centers for Disease Control and Prevention (CDC).

Surveys can be an important part of your evaluation efforts. As evaluators we have probably all heard (or said), “We need a survey!” Before launching into any survey effort make certain you know the purpose of the proposed survey. To determine your purpose, take time to answer these four simple questions:

• Hot Tip #1:  Why a survey? Is there another way—focus groups, direct observation, document review, or secondary data sources—that will provide you with the information you need for your evaluation?
• Hot Tip #2:  Who do you intend to survey?  Are you surveying program staff, partners, stakeholders, recipients, employers, providers, etc.?
• Hot Tip #3:  What do you need to know? You may have a long wish list of things you would like to know—go through your list carefully and determine which ones are “need to know” rather than simply “nice to know.”  It is important that you take the time upfront to determine if each question you are considering is absolutely essential.

A useful check can be to ask yourself:

If I know________ (fill in the blank with the information you hope to gather through the survey), I will be able to ________ (fill in the blank with what you hope to be able to do, for example measure a specific outcome).

• Hot Tip #4:  How will the survey be administered (i.e., telephone, in-person, Internet)? Your timeframe and available resources will likely affect which mode you select.  Also, consider which mode will work best for the people you intend to survey.

Rad Resources

• Program Evaluation Tip Sheet: Constructing Survey Questions A tip sheet that we developed for funded programs through the Division for Heart Disease and Stroke Prevention at CDC
• Spacing, Position, and Order:  Interpretive Heuristics for Visual Features of Survey Questions a paper from Tourangeau, Couper, and Conrad at the University of Michigan: Institute for Social Research
• Question Wording guidance from the American Association for Public Opinion Research
• Analyzing Polls: Interpreting Analysis-Questionnaire Design from the Roper Center Public Opinion Archives
• Standards and Guidelines for Statistical Surveysfrom the Whitehouse Office of Management and Budget
• Survey Research on the Research Methods Knowledge Base

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.