We are Linda Cabral and Laura Sefton from the University of Massachusetts Medical School’s Center for Health Policy and Research, and part of the PCMHI Evaluation Team whose work is being described all week. We want to share our team’s experience of being participant observers at ‘learning sessions’ and how it’s helped our overall multi-modal data collection efforts.
Staff from the 45 primary care practices in the PCMHI took part in seven, day-long learning sessions over the course of the 3-year initiative. These sessions offered technical assistance to practice staff by bringing in experts in PCMH implementation and provided opportunities for practices to learn from each other and share lessons learned. Through attendance, our team was able to observe the content being presented as well as the participants’ reactions to it, thereby giving us a better understanding of what practices are working on in their transformation to becoming medical homes. Additionally, our observations helped us to shape an interview guide for future site visits with the practices.
Hot Tips: Use a template to collect data in a standardized way. Each activity during a learning session had handouts or a PowerPoint presentation that contained information for the attendees. We developed a template to collect other relevant data from each session, which encompassed 3 mains areas:
Methodology Notes – What is the format of the activity, e.g., panel presentation, group activity? Who are the people leading the activity?
Field Notes – What is happening in the activity? Who is in the audience? What is the level of participant engagement? What types of questions are being raised? How are these questions being answered?
Personal Notes – What are your (the evaluator’s) impressions of the activity?
Use the opportunity to network with attendees. Explain why you, as an evaluator, are attending the session. Get participants’ thoughts on what would be important to evaluate. They may have ideas you hadn’t considered to shape future data collection questions. We used their ideas in developing our interview guide for future site visits.
Hold an internal team debriefing meeting after each event. These meetings allowed the evaluation team to share information with each other so that we could all have an understanding of what happened during all activities at each learning session.
Lesson Learned: Attending the learning sessions gave the team a frame of reference that was valuable to completing future site visit interviews that were conducted as part of the evaluation. When interviewees referenced the learning sessions, the interviewer’s prior knowledge allowed for a mutual understanding and helped build rapport.
Rad Resource: This article from on the online journal Qualitative Social Research describes in more detail participant observation as a data collection tool.
The American Evaluation Association is celebrating Massachusetts Patient-Centered Medical Home Initiative (PCMHI) week. The contributions all this week to aea365 come from members who work with the Massachusetts Patient-Centered Medical Home Initiative (PCMHI). Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.