I am Carla Hillerns from the Office of Survey Research at the University of Massachusetts Medical School’s Center for Health Policy and Research. In 2015, my colleague and I shared a post about how to avoid using double-barreled survey questions. Today I’d like to tackle another pesky survey design problem – leading questions. Just as we don’t want lawyers asking leading questions during a direct examination of a witness, it’s also important to avoid leading survey respondents.
By definition, a leading question “guides” the respondent towards a particular answer. Poorly designed questions can create bias since they may generate answers that do not reflect the respondent’s true perspective. Using neutral phrasing helps uncover accurate information. Here are a few examples of leading questions as well as more neutral alternatives.
Hot Tips for Avoiding Leading Questions:
- Before deciding to create a survey, ask yourself what you (or the survey sponsor) are trying to accomplish through the research. Are you hoping that respondents will answer a certain way, which will support a particular argument or decision? Exploring the underlying goals of the survey may help you expose potential biases.
- Ask colleagues to review a working draft of the survey to identify leading questions. As noted above, you may be too close to the subject matter and introduce your opinions through the question wording. A colleague’s “fresh set of eyes” can be an effective way to tease out poorly phrased questions.
- Test the survey. Using cognitive interviews is another way to detect leading questions. This type of interview allows researchers to view the question from the perspective of the respondent (see this AEA365 post for more information).
Rad Resource: My go-to resource for tips on writing good questions continues to be Internet, phone, mail, and mixed-mode surveys: The tailored design method by Dillman, Smith & Christian.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to firstname.lastname@example.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.