We are Debi Lang and Kathy Muhr, members of the Research and Evaluation Unit at the University of Massachusetts Medical School Center for Health Policy and Research.
Populations considered hidden or hard to reach for participation in qualitative evaluation studies may be small in size, their members difficult to locate, or hard to distinguish from general populations. In their article, Salgalnick and Heckathorn state such groups historically include subjects in HIV/AIDS research but can include undocumented immigrants, or the homeless.
Evaluations that rely on data from hidden or hard to reach populations present challenges when names and contact information do not exist, are not accessible, or are generated in a way that may introduce biased results. In two recent projects, we used approaches to identify 1) family members of Hospice patients who had died; and 2) adults with mental health conditions who are deaf/hard of hearing (D/HH) or Latino.
Hot Tip: Avoid Bias
- For the Hospice project, we used claims and enrollment data to identify family members of Hospice decedents, rather than request the information from Hospice providers. This approach avoided a potentially biased sample of family members who were predominantly satisfied with their services.
Hot Tip: Hire Cultural Brokers
- To identify D/HH or Latino adults with a mental health condition, we hired cultural brokers, with the experience and language of the groups we wished to contact. As peers and integral members of our evaluation team, the cultural brokers helped to identify group members and create a viable sample of potential participants.
- To recruit cultural brokers, we made announcements at various stakeholder and committee meetings, brought copies of the job description, and brainstormed with attendees to identify likely candidates.
Hot Tip: Maintain Confidentiality
- Whether gathering names and contact information of potential study participants from a database or by word-of-mouth, use compliance procedures to maintain confidentiality of personal information and to protect their rights. Both projects required approvals from either an Internal Review Board (IRB) or Compliance Unit to identify and recruit participants.
Lessons Learned: Budget Wisely
- To budget a project which identifies hidden populations, consider the time needed to generate the study sample, including IRB and data access approval.
- Consider costs for hiring cultural brokers and/or translators, as well as for participant incentives, travel, and costs associated with rescheduling meetings. These expenses support successful recruitment and data collection activities.
RAD Resources: The hyperlink above and the following resources discuss sampling designs used to identify hidden or hard to reach populations.
- Accessing Hidden and Hard-to-Reach Populations: Snowball Research Strategies.
- A Guide for the Design and Conduct of Self-Administered Surveys of Clinicians.
- A Venue-Based Method for Sampling Hard-to-Reach Populations.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
We’ve learned a lot from Ben Van Hunnik at Prodigy Research, who strongly emphasizes building a deep understanding of potential research subjects so you can not only find them in the first place, but follow-up with them over time for your study. He blogs at http://prodigyresearch.com/.