I am Humberto Reynoso-Vallejo a Director for Program Evaluation with the Center for Health Policy and Research at the University of Massachusetts Medical School.
Conducting qualitative research for evaluation purposes can be complex and often time consuming, particularly when dealing with large amounts of data. The use of Summative Analysis is useful technique to employ when working in teams. Summative Analysis is a collaborative analytic technique based on consensus-building activities that includes a range of insights and reflections from all members of a team of researchers. The main purpose is to identify the essential text, defined as the elements offering a point of entry into the meaning of whole text, without which we might not understand the data. Summative Analysis is a journey towards understanding. Key aspects include consensus of opinion, is highly participatory, multidisciplinary, considers content and context, and builds understanding. The goal is to develop individual and, eventually, group paragraphs describing the essential text. We recently used this technique in one of our larger projects, the evaluation of the Patient-Centered Medical Home Initiative in Massachusetts, and found it extremely useful.
Hot tips:
Step 1. Start by developing individual paragraphs of raw material by each research team member, identifying the essential text, by capturing fundamental elements of the data. This is a first attempt to understand the data, and each team member writes notes, bullets, memos, or verbatim text.
Step 2. The team meets to discuss individual paragraphs, and orders data based on group findings. Data outputs are divided by themes or topic-oriented headings. This exercise allows team members to start thinking about a hierarchical map of the data. It is important to avoid interpreting the data at this stage. Organize the data by creating a matrix that includes the name of the team member in each column and pastes their respective notes. The result is a table that can be read as: member A mentioned this, member B mention this, whereas member C mention that, etc.
Step 3. Team meets again and jointly develops a group paragraph that synthesizes their work, refine emergent themes, confirm thematic or topic-oriented headings, and order key concepts and categories. The result is a paragraph that can be read as: everyone mentioned this, some team members mentioned that, and one or two (outliers) mentioned this.
Step 4. A final group working session requires that all team members have a good grasp of the text and its significant aspects. In this stage, the team conducts an analysis and interpretation of elements of the group paragraph. The team also explores what ideologies, theories, or conceptual frameworks were used to arrive at that group paragraph.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.