Hi, I’m Ama Nyame-Mensah. I am a doctoral student at the University of Pennsylvania’s School of Social Policy & Practice. In this post, I will share with you some lessons learned about incorporating demographic variables into surveys or questionnaires.
For many, the most important part of a survey or questionnaire is the demographics section. Not only can demographic data help you describe your target audience, but also it can reveal patterns in the data across certain groups of individuals (e.g., gender, income level). So asking the right demographic questions is crucial.
Lesson Learned #1: Plan ahead
In the survey/questionnaire design phase, consider how you will analyze your data by identifying relevant groups of respondents. This will ensure that you collect the demographic information you need. (Remember: you cannot analyze data you do not have!)
Lesson Learned #2: See what others have done
If you are unsure of what items to include in your demographics section, try searching through AEA’s Publications or Google Scholar for research/evaluations being done in a similar area. Using those sources, you can locate links to specific tools or survey instruments that use demographic questions that you would like to incorporate into your our work.
Lesson Learned #3: Let respondents opt out
Allow respondents the option of opting out of the demographics section in its entirety, or, at the very least, make sure to add a “prefer not to answer” option to all demographic questions. In general, it is good practice to include a “prefer not to answer” choice when asking sensitive questions because it may make the difference between a respondent skipping a single question and discontinuing the survey altogether.
Lesson Learned #4: Make it concise, but complete
I learned one of the best lessons in survey/questionnaire design at my old job. We were in the process of revamping our annual surveys, and a steering committee member suggested that we put all of our demographic questions on one page. Placing all of your demographic questions on one page will not only make your survey “feel” shorter and flow better, but it will also push you to think about which demographic questions are most relevant to your work.
Collecting the right demographic data in the right way can help you uncover meaningful and actionable insights.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
When demographic questions are listed on a survey, I would encourage evaluators to put those on at the END of the survey in order to avoid stereotype threat. There is sufficient research on this: putting those questions at the beginning of a survey can influence how people answer the questions.
Annie – I agree that surveys should not be too long in length. However, I caution against creating a short survey just for the sake of having a short survey. You run the risk of excluding questions that really should be included. (And I would say that it is bad practice!) That said, I completely agree with you that ‘nice to know’ questions should be excluded at all costs!
Darcy – I am glad you enjoyed my post! I agree that many people overlook the importance of demographic questions. (However, much of this, as you stated, is due to people’s lack of understanding of demographic and social processes.) I recently started a blog (anyamemensah.weebly.com/blog) because I wanted to share what I have learned over the years about survey research, program evaluation, and statistics. You may find some of my upcoming posts useful. Good luck with your studies!
Ama,
I really enjoyed your post. As a student who is new to the research side of things, I have more of a respect for the consideration that goes into creating surveys and the importance of demographic data. I feel that many people don’t understand the importance of the demographic questions and that it leads to much confusion. I like the idea of keeping all the demographic questions on one part of the survey. I will consider your tips in my upcoming classes.
Darcy
Something else to consider. Surveys need to be as short as possible so that people don’t get bored and start giving you bad data. With that in mind, if you don’t have a specific reason to ask a question and you know you won’t DO anything with the answers, then don’t include the question. Avoid ‘nice to know’ questions at all costs!