Hi everyone! Carolyn Fisher, Martina Todaro, and Leah Zallman of the Institute for Community Health here, with the second installment in our epic tale of survey design.
When last we left our heroes, we had arrived at this question to measure vulnerability due to low income, and were about to pilot it with our client’s participants.
Do you get ANY of the following benefits? SNAP/Food stamps, WIC, SSI, SSDI, TANF, Housing Assistance, Medicaid
- Yes
- No
- Unsure
- I don’t want to answer
Cool Trick: Pilot your surveys! Have volunteers fill out the survey, read each question back, and explain why they gave each answer. We always learn when we do this!
In this case, our piloting gave us an unexpected result: our client and their participants felt that this question wouldn’t work due to the stigma around public benefits — stigma would prevent people from answering. Further, they felt some people taking the survey could be undocumented immigrants, who, even when eligible for benefits, are often reluctant to apply.
We weren’t quite ready to let go of this question, though. In trying to tweak it to address these issues, we came up with:
Just based on your income, are you or would you be eligible for any of the following benefits? Food stamps, cash assistance, housing assistance, or [state specific Medicaid name]?
- Yes
- No
- I don’t know
- I don’t want to answer
I hope everybody reading this is cringing! Perhaps the less said about this version, the better.
Soon coming to our senses, we decided to re-focus on the actual domain we were trying to assess: vulnerability. The following is based on validated measures of social determinants of health.
In the past 12 months, have any of the following been true? (check all that apply)
- You worried whether your food would run out before you got money to buy more
- You worried about losing housing or were homeless
- The electric, gas, water or oil company threatened to shut off services in your home
- None of the above
- I don’t want to answer
Rad Resource: Health Leads USA has published a toolkit with validated and unvalidated measures for domains of social needs and social determinants of health.
This final question both:
-Measures the actual construct we needed to know about (vulnerability)
-Is responsive to the contexts where it is fielded
We hope that hearing about our journey with this question was helpful to you in your own work — happy voyaging!
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.