ACA TIG Week: Don Glass on Seeing the System Using Root Cause Analysis

Hi. I am Don Glass, an independent consultant who is a visual artist, learning designer, and developmental evaluator.

Hot Tips: One of the things that I love about attending the AEA annual conference is getting the opportunity to better understand how my work can relate to and be informed by recent debates and developments in the field. For example, Michael Quinn Patton suggested at the 21st Empowerment Evaluation anniversary session that EE should more actively consider a Complexity Perspective and Systems Thinking. This got me reflecting on being more aware of how people and programs operate in complex systems.

It also provided me with some conceptual language for my own session for the Arts, Culture, and Audiences TIG to talk about Root Cause Analysis (RCA).

Lessons Learned: I used Root Cause Analysis as part of the developmental evaluation work that I am doing for an arts museum education program that was integrating visual art and STEM. A recent external evaluation had shown limited impact on student learning. I was contracted to use a more collaborative design and evaluation approach to inform and support the redesign of the education program.

During the exploratory phase, I interviewed staff, observed teaching, and analyzed curricular materials. Based on this initial qualitative data and the findings from the external program evaluation, I conducted a RCA to map the various problems of the program and its contexts.

The RCA was visually displayed as a fishbone diagram to help us select and focus on a manageable set of improvement areas (e.g., weak curricular connections, overly complex activities, and no assessment feedback), as well as see where these areas fit into and possibly influence the rest of the system (e.g., teacher engagement). This visual display of inter-connected problems and conditions, helped us to “see the system” and begin to not only understand why it was producing lackluster outcomes, but how we might strategically develop and test new approaches.

Rad Resources: Michael Quinn Patton’s chapter The Developmental Evaluation Mindset: Eight Guiding Principles in the new book Developmental Evaluation Exemplars: Principles in Practice.

The chapter See the System That Produces the Current Outcomes in Learning to Improve by the folks at the Carnegie Foundation for the Advancement of Teaching who are exploring Improvement Science and Networked Improvement Communities in education.

Tools for viewing systems and processes in the encyclopedic reference The Improvement Guide: A Practical Approach to Enhancing Organizational Performance.

The American Evaluation Association is celebrating Arts, Culture, and Audiences (ACA) TIG Week. The contributions all week come from ACA TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.