My name is Susan Keskinen. I work for Ramsey County (Minnesota) Community Human Services as a Senior Program Evaluator. My evaluation projects are related to employment and supportive services provided to Minnesota Family Investment Program (MFIP) participants, Minnesota’s version of the Temporary Assistance to Needy Families (TANF) program. I am the Communications Chair for the Minnesota Evaluation Association.
Hot Tip:
My biggest challenge with qualitative research has been the analysis of the data. I have developed the following process that enables me to do it systematically and effectively.
- Read the notes from three to five interviews and determine a preliminary set of themes.
- Create a ‘theme’ document in Word with each theme listed on a separate page.
- Create a ‘working’ copy of the notes from each interview in Word and give the text of each of those ‘working’ copies a unique color and/or style of font.
- Read each ‘working’ copy document and paste a comment related to a theme from the ‘working’ copy to the appropriate theme in the ‘theme’ document. (By using ‘delete’ and ‘paste’ instead of ‘copy’ and ‘paste’, you can keep track of the portion of an interview that you have not been able to attribute to any existing theme.)
- After going through all ‘working’ copy documents once, read the portions of the notes that did not fit into any preliminary theme and determine what additional themes to add. Add those themes to the ‘theme’ document and do Step 4 again.
The result is one Word document that lists all themes and the specific comments (in various colors and styles of font) related to each theme. The different types of font make it easy to count the number of unique people who gave a comment related to a theme. It is also easy to combine themes and know how it affects the number of individual comments.
If you want to analyze the data by different groups of interviewees, give all interview notes in a group the same color font but give each individual in that group a distinctly different style of font. As shown in the example below, five different people from three different groups (red, green and blue) gave comments about Communication.
The American Evaluation Association is celebrating Minnesota Evaluation Association (MN EA) Affiliate Week with our colleagues in the MNEA AEA Affiliate. The contributions all this week to aea365 come from our MNEA members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Great write up of a rather daunting and sometimes complex procedure.
Thanks Susan. I’ve been using a similar method for many years, so it’s wonderful to know that I’m not alone. 🙂 Sometimes I just use my generous supply of magic markers (low technology) and then summarize in a database. Here’s a summary of an intensive community input process done for the purpose of developing a comprehensive health plan in our county. This 113 page document summarizes 32 key stakeholder interviews, 22 focus group, and 10 community forums: http://www.whatcomcounty.us/health/pdf/chp_summaryofpublicinput.pdf Based on this summary I came up with a 1-page vision map where related issues are colorcoded or underlined or linked with sympbols to show how they are related across the 4 main themes: health promotion, healthy environment, communicable disease, and health care delivery. Here’s the vision map link: http://www.whatcomcounty.us/health/pdf/chp_vision_map.pdf
I think this is a fantastic idea and a very creative way to handle what often feels like an overwhelming task. The one challenge would be for studies that involve a LOT of interviews – coming up with the different fonts and then also being able to discriminate among them visually…e.g., my biggest challenge with QDA came in a study involving 68 interviews. I used a QDA software package, but it was still pretty intense managing the mass of text data.
Thank you for sharing your method!