Hello. We are Valerie Konar, Carla Hillerns, and Michelle Landry from the University of Massachusetts Medical School’s Center for Health Policy and Research. Today, we share lessons learned from our evaluation work for the MA Patient Centered Medical Home Initiative.
The strength of many evaluation designs includes the use of a rigorous control group. However, identifying practices that received no exposure to medical home interventions was not possible as most practices in Massachusetts were exposed to some form of medical home knowledge. We therefore needed to secure a set of comparison practices that may be involved in medical home activities but not be receiving the same level of intervention as our study practices. Recruiting member practices for a comparison group and keeping them engaged over several years presented unique challenges.
How do you entice a busy primary care practice to sign-on and complete the tasks requested of them as part of the comparison data collection process with little or no compensation?
Hot Tip: Network! Use professional organizations and contacts to spread the word and encourage participation.
- Reach out to practices that initially showed interest in the intervention portion of the project, but were not selected; they may be interested in participating in a different way.
- Vary and repeat your recruiting efforts until you generate the necessary interest.
- Explain WHY participation is so important.
Hot Tip: Offer feedback on the results of practices’ efforts as compensation. This feedback can be used as quality improvement tools or relate to other organizational goals.
- If budgets allow, offer some form of compensation (e.g., small stipends) in acknowledgement of time and effort. Incrementally increase the stipend value over time to help encourage motivation to stay the course.
Engagement through the end of the project is key to successful comparison analyses. During the project’s life, how do you maintain comparison group’s participation?
Hot Tip: Relationships are key! Simplifying your point of contact will eliminate confusion. Assigning one contact person who is knowledgeable and accessible will go a long way to maintaining relationships.
Hot Tip: Try to time requests so as not to coincide with busy periods.
- Bundle requests when possible to minimize the number of communications.
- Make deliverables easy to complete and accommodate requests, if possible (e.g., allow responses by mail and web).
- Predictability helps! Provide advance reminders for task assignments
Lesson Learned: Being mindful of what groups are able to provide. Being sensitive to the amount of time a task takes will increase your chance of receiving the necessary data.
Rad Resource: RealWorld Evaluation: Working Under Budget, Time, Data and Political Constraints offers strategies for minimizing selection bias in a real-world context.
The American Evaluation Association is celebrating Massachusetts Patient-Centered Medical Home Initiative (PCMHI) week. The contributions all this week to aea365 come from members who work with the Massachusetts Patient-Centered Medical Home Initiative (PCMHI). Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
The MA Patient-Centered Medical Home Initiative was a demonstration project. All practices, whether intervention or comparison, were required to apply and be selected to participate in the demonstration. The intervention sites were given technical assistance including access to medical home facilitators for quality improvement, participation in learning collaborative educational activities and assistance with using electronic health record systems during the demonstration, whereas the Comparison sites were not given any interventions during the demonstration.
Some intervention practices received financial assistance for their participation in the project, which included evaluation activities as part of the scope of work. Comparison sites were given small stipends linked to the completion of each of the data collection activities. The Evaluation Team simplified data collection by offering formatted data collection tools for new information, as well as referring to the past data collected for updates as a reminder and starting point when the task was repeated.
During the solicitation process and contract negotiation, the contracts created allowed for all practices to terminate their agreement easily. Only 2 practices out of 49 intervention sites and 3 practices out of 22 comparison sites terminated their participation.
In regards to the data collection process for medical homes. Do you offer any type of assistance to physician’s practices in regards to data collection and organization of data? Do physicians practices whom choose to participate in the Medical Home Model have the opportunity to ask for help managing data collection and if the process turns out to be overwhelming for physicians are they able to terminate there Medical Home Model and agreement easily. I would think if you offered assistance with the paperwork and data collection and an easy termination option to a busy primary care practice more physician’s offices may participate.