Hello, AEA365 community! Liz DiLuzio here, Lead Curator of the blog. This week is Individuals Week, which means we take a break from our themed weeks and spotlight the Hot Tips, Cool Tricks, Rad Resources and Lessons Learned from any evaluator interested in sharing. Would you like to contribute to future individuals weeks? Email me at AEA365@eval.org with an idea or a draft and we will make it happen.
My name is Liane Ventura, MPH, and I serve as a faculty Research Associate with the Center for Applied Research and Evaluation (CARE) in Women’s Health at East Tennessee State University (ETSU). I am an applied Public Health researcher who is interested in mixed methodology and community-engaged approaches.
A health equity driven approach to program evaluation includes qualitative data components. Qualitative data for program evaluation can be generated through various mechanisms such as key informant interviews, focus groups, or open-ended survey responses. These data provide rich context to complement a quantitative evaluation and may also stand on its own.
Qualitative data may also provide important context for understanding individuals’ lived experiences and perceptions in the context of various programmatic elements. Given that programmatic interventions in the Public Health field often focus on reducing health inequities and health disparities, it is important to understand the rich context surrounding the process of implementation as well as the outcomes. As such, qualitative data helps to fill this gap through providing the “why” and the “how” of programmatic intervention.
While traditional qualitative methods include lengthy analysis processes such as double coding and interrater reliability, the Rapid-Turn Around method offers a systematic and reliable alternative. Rapid methods occur while data are being collected, are explanatory in nature, and may lend to further in-depth analysis. At CARE Women’s Health at ETSU, we conduct a rapid coding process for our key informant interview data prior to conducting a formal qualitative coding process.
In addition to being a time saver, the Rapid Turn-Around method is easily accessible. Rather than using industry software such as NVivo, MAXQDA, or Atlas.Ti, the rapid approach relies on systematically producing qualitative results using a summary matrix via Microsoft Excel. The summary matrix has one column with “domains”, which are neutral representations of the interview guide questions and/or overarching theoretical constructs. Each subsequent column will represent one interview.
With the rapid method, summarizing data within each domain takes the place of coding. So, if you are summarizing a sample of interviews as a team, it is important to first develop consistency with how you are summarizing. For example, that every coder is highlighting the same main ideas consistently and/or emphasizing the same level of nuance within each domain, for each interview. Thus, it is important to meet throughout the rapid coding process to ensure harmony among summary styles.
The rapid summary matrix will result in a full and complete spreadsheet of summaries for each domain (associated with the interview guide questions) and for each interview (represented via each column). Thus, each row will contain a series of summaries for each domain. The analytic approach is to review the summaries for each domain (across each row) and develop a high-level overarching executive summary. It is possible to thereafter use the matrix to develop a detailed codebook for continuing with a formal qualitative analysis.
The Rapid Turn-Around method is a quick and efficient way to systematically categorize qualitative data. We have successfully published social media posts, reports, and peer-reviewed manuscripts using this approach. Thank you for your interest in applying qualitative methods for program evaluation! I am happy to connect with you to continue the discussion.
Rad Resources
Some examples of where our team has applied the rapid coding process:
- Contraceptive counseling during the COVID-19 pandemic (2020) at Choose Well participating clinical partners in South Carolina
- Telehealth for contraceptive care during the initiative months of the COVID-19 pandemic at local health departments in 2 US states: A mixed-methods approach
- Contraceptive care service provision via telehealth early in the COVID-19 pandemic at rural and urban federally qualified health centers in 2 southeastern states
Additional readings and trainings:
- Qualitative methods in implementation research: An introduction
- Rapid Techniques in Qualitative Research: A Critical Review of the Literature
- Rapid Turn-Around Qualitative Research: Strategic Decision-making for Rigor and Feasibility
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.