Hi, I’m Ann Lawthers, Principal Investigator for the evaluation of the Massachusetts Patient-Centered Medical Home Initiative (PCMHI), and faculty in the Department of Family Medicine and Community Health at the University of Massachusetts Medical School (UMMS). This week, the UMMS PCMHI Evaluation Team will be presenting a series of Hot Tips, Lessons Learned and Rad Resources for evaluating large, complex and multi-stakeholder projects. We cover issues that surfaced during the design, data collection and analysis phases.
Hot Tip: When beginning to think about the design of a large, complex project, consider using a mixed methods approach to maximize the breadth and depth of evaluation perspectives. The Massachusetts PCMHI’s principal stakeholders – state government officials, insurers, and the practices themselves – have invested time and financial resources in helping primary care practices adopt the core competencies of a medical home. Each stakeholder group came into the project with different goals and agendas.
We selected a mixed methods evaluation approach to answer three deceptively simple questions:
- To what extent and how do practices transform to become medical homes?
- To what extent and in what ways do patients become active partners in their health care?
- What is the initiative’s impact on service use, clinical quality, patient and provider outcomes?
Lesson Learned: Our mixed methods approach allowed us to tap into the perspectives and interests of multiple stakeholder groups. The primary care practices participating in the PCMHI demonstration were keenly interested in the “how” of transformation (Question 1) while state policy makers wanted to know “if” practices transformed (also Question 1). We addressed the “how” principally through qualitative interviews with practice staff and the TransforMED Medical Home Implementation Quotient (MHIQ) questionnaire, completed by practice staff.
Participating practices also cared a great deal about the initiative’s affect on patients. Did patients perceive a change and become more actively involved in their health care (Question 2)? We used patient surveys to address this question.
Finally, all stakeholder groups were interested in the impact question (Question 3). Claims data, clinical data reported by practices, staff surveys and patient surveys all provided different views of how the PCMHI affected service use, clinical quality and other outcomes.
The American Evaluation Association is celebrating Massachusetts Patient-Centered Medical Home Initiative (PCMHI) week. The contributions all this week to aea365 come from members who work with the Massachusetts Patient-Centered Medical Home Initiative (PCMHI). Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.