Hello. We are Valerie Konar, Carla Hillerns, and Michelle Landry from the University of Massachusetts Medical School’s Center for Health Policy and Research. Today, we share lessons learned from our evaluation work for the MA Patient Centered Medical Home Initiative.
The strength of many evaluation designs includes the use of a rigorous control group. However, identifying practices that received no exposure to medical home interventions was not possible as most practices in Massachusetts were exposed to some form of medical home knowledge. We therefore needed to secure a set of comparison practices that may be involved in medical home activities but not be receiving the same level of intervention as our study practices. Recruiting member practices for a comparison group and keeping them engaged over several years presented unique challenges.
How do you entice a busy primary care practice to sign-on and complete the tasks requested of them as part of the comparison data collection process with little or no compensation?
Hot Tip: Network! Use professional organizations and contacts to spread the word and encourage participation.
- Reach out to practices that initially showed interest in the intervention portion of the project, but were not selected; they may be interested in participating in a different way.
- Vary and repeat your recruiting efforts until you generate the necessary interest.
- Explain WHY participation is so important.
Hot Tip: Offer feedback on the results of practices’ efforts as compensation. This feedback can be used as quality improvement tools or relate to other organizational goals.
- If budgets allow, offer some form of compensation (e.g., small stipends) in acknowledgement of time and effort. Incrementally increase the stipend value over time to help encourage motivation to stay the course.
Engagement through the end of the project is key to successful comparison analyses. During the project’s life, how do you maintain comparison group’s participation?
Hot Tip: Relationships are key! Simplifying your point of contact will eliminate confusion. Assigning one contact person who is knowledgeable and accessible will go a long way to maintaining relationships.
Hot Tip: Try to time requests so as not to coincide with busy periods.
- Bundle requests when possible to minimize the number of communications.
- Make deliverables easy to complete and accommodate requests, if possible (e.g., allow responses by mail and web).
- Predictability helps! Provide advance reminders for task assignments
Lesson Learned: Being mindful of what groups are able to provide. Being sensitive to the amount of time a task takes will increase your chance of receiving the necessary data.
Rad Resource: RealWorld Evaluation: Working Under Budget, Time, Data and Political Constraints offers strategies for minimizing selection bias in a real-world context.
The American Evaluation Association is celebrating Massachusetts Patient-Centered Medical Home Initiative (PCMHI) week. The contributions all this week to aea365 come from members who work with the Massachusetts Patient-Centered Medical Home Initiative (PCMHI). Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to firstname.lastname@example.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.