AEA365 | A Tip-a-Day by and for Evaluators

Aug/16

3

Ruth Aboagye, Sai Cherala, Pam Senesac, and Joan Johnston on Quality Improvement (QI) in Evaluation: Ask Why Again and Again and Again

Hello again from the MBA, doctor and two nurses at UMass Medical School’s Center for Health Policy and Research – Office of Health Care Innovation and Quality to talk about “5 Whys” as an evaluation tool. We have all encountered problems that refuse to go away. We think we’ve “fixed” it and it comes back or we find we’ve only fixed part of it. Stubborn and recurring problems are often symptoms of deeper problems. Evaluate the symptoms to diagnose the root cause in order to solve the problem effectively and permanently. Sakichi Toyoda’s 5 Whys technique – a simple but powerful tool – can help peel away the symptomatic layers and lead to the root cause.

This technique simply asks “why” until you reach the underlying source of the problem. Then, a robust counter-measure often becomes apparent. You’ll know that you reached the source problem when you no longer have answers to the “why” question.

See how the “5 Whys” was used to save the Washington Monument. Turned out the countermeasure was as simple as turning the lights on after dusk.

Sefton“5 Whys” Key Points:

  • A simple, practical tool that does not require statistical analysis or hypothesis testing, and can often be completed without data. It’s flexible and combines well with other methods and techniques.
  • It uses “counter-measures” rather than “fix and be done” solutions. Counter-measures seek to prevent the problem from recurring.
  • 5 Whys is the rule of thumb. You may need fewer or more ‘whys’ than five to get to the root cause.

Hot Tips:

  • Follow Toyota’s “Go and See” philosophy. Go where the problem is happening to gain an in-depth understanding of the problem.
  • Write down the source problem. This helps formalize and describe the problem completely; helps the team focus on the problem.
  • Every time you ask “why,” look for answers that are factual – not events that might have happened. This prevents the 5 Whys becoming just a process of deductive reasoning.
  • Write down the questions and answers in full sentences so that everyone is on the same page.
  • Be careful not to rely only on 5 Whys for critical or more complex problems; complement with other tools such as Cause and Effect (Fishbone) diagrams.

Lessons Learned:

  • Include subject matter experts in the root cause analysis dialogue
  • Avoid attributing blame; the goal is to get to the systemic cause
  • Beware of intentional or unintentional influence during the 5 Whys exercise
  • 5 Whys is best suited for simple to moderately-difficult problems

Rad Resource:

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

6 comments

  • Mark Graban · December 7, 2016 at 2:54 pm

    Root cause analysis, including asking why multiple times is powerful.

    As I’ve blogged about here, you have to do more than ask why:

    http://www.leanblog.org/2015/11/you-just-have-to-ask-why-five-times-lean-or-leanstartup-requires-more-than-that/

    And five isn’t a magic number:

    http://www.leanblog.org/2015/11/maybe-we-should-call-it-the-many-whys-instead-of-the-five-whys/

    The story about the Washington Monument isn’t exactly 100% true, even if it makes for a fun example:

    http://thekaizone.com/2014/08/5-whys-folklore-the-truth-behind-a-monumental-mystery/

    Reply

  • Nicole Campbell · August 18, 2016 at 11:38 pm

    Hello Ruth, Sai, Pam and Joan!

    It looks as though your article has been popular with myself and my classmates. My name is Nicole Campbell and I am also a candidate in the Professional Masters of Education at Queens University.

    Let me start off by telling you a bit about my teaching practice. I am a post-secondary educator and I teach at Western University in the medical sciences program. I was recently hired as a faculty member and the first year has been an amazing experience. I have been fortunate enough to be part of implementing some changes in our program, which as significantly impacted myself as an educator.

    The reason why your article caught my attention is because I can picture so many ways that asking these “whys” would be so helpful. As I mentioned, we have been proposing some changes in our program and as we have gone through this process, we are often faced with a few road blocks. I never would have thought to apply this process of asking why until you can’t anymore to try and get at the root of the problem but it makes so much sense.

    When I think about how I could apply this to my own teaching practice, I can absolutely see the benefit of this process but I also had a few questions. Typically, before any major changes are made in our program, we have run them by a group of individuals that includes other faculty and administrative staff. I know you mentioned including experts in this process but I am also wondering at what point have you found there are too many cooks in the kitchen? I have found that depending on the role that the experts play in the program, they can have different perspectives or even goals in mind. I think that having multiple view points is always a good idea but I wondered if you could comment on any tips or best practices when involving multiple stakeholders?

    Thanks so much.

    Nicole Campbell, PhD
    Assistant Professor
    Western University

    Reply

  • Jason Lane · August 11, 2016 at 12:16 pm

    Hello Ruth, Sai, Pam, and Joan.
    Thanks for this great post and the rad resources. My name is Jason Lane, I’m a Master of Education student studying through Queen’s University in Kingston, Ontario. I’m currently taking a course in Program Evaluation, and as such have been introduced to the AEA365 Blog.

    Your post intrigued me as I had not seen this evaluation method used before. Asking the “5 Whys” is such an easily applicable tool to so many situations, and I have already begun to develop ways to use it professionally when discussing medical case studies and patient outcomes.

    Checking out your rad resources, I was interested by the points of view. I find myself with reservations about the Harvard approach that behind every technical problem is a human error. I read critiques of the 5 Whys that suggested this creates an environment of blame seeking instead of trying to source a systems or procedural problem, I’d be very curious to hear your thoughts to this.

    I also find myself questioning how often do you find that a different group may come to a different solution for the same problem? Would it be seen that one group perhaps missed a critical component when compared? For example, the puddle on my floor yesterday… Why: Because the window was left open and it rained, OR because there is no awning above the window to protect from a driving rain. Each of these could then continue to produce their own continuation of the Whys, do you find this correct use of the tool, or in your experience is it over-complicating things?

    Once again, thanks for sharing this, I look forward to learning more about it and putting it to use.

    -Jason

    Reply

    • Mark Graban · December 7, 2016 at 5:07 pm

      To Jason:

      You said “I also find myself questioning how often do you find that a different group may come to a different solution for the same problem? Would it be seen that one group perhaps missed a critical component when compared?”

      Sure, different groups may go down different paths. That’s fine.

      What matters in this type of problem solving is the experimental process of Plan Do Study Adjust. Do we ideally want the “right” answer the first time? Sure.

      But when any group goes through root cause analysis, the best we have is a hypothesis about a root cause or causes. You don’t determine root cause by talking about it.

      You frame up different countermeasures that you think will address the root cause and, therefore, the underlying problem. You test countermeasures and you might learn, “Nope, that wasn’t really a root cause” or “That wasn’t the right countermeasure.”

      It might take two different groups different paths or amounts of time to get there, but they should get to the same place… or equally good countermeasures.

      Reply

    • Mark Graban · December 9, 2016 at 9:55 am

      The point of 5 whys, in a Lean culture, is NOT to blame individuals.

      If somebody is doing the 5 whys and that’s creating a culture of blame, they’re doing it wrong.

      A Lean organiation realizes they need to help prevent human error by designing systems that make it more difficult for error to occur (the idea of error proofing).

      Reply

  • Gordon Laffin · August 9, 2016 at 12:40 pm

    Dear Ms. Aboagye, Cherala, Senesac, and Johnston,

    Hello, my name is Gordon Laffin and allow me to briefly introduce myself. I am currently studying my Professional Masters of Education at Queen’s University in Kingston, Ontario. I am a teacher with 9 years of Middle and High School experience.

    I had the pleasure of reading your recent post to aea365.org and found your simple and effective “5 Whys” method very intriguing. As I am a teacher, it made me think of when a younger student repeatedly asks why after you ask him or her to do something. As I read through your Rad Resources and watched the Harvard Business Review video, I realized the link. Sometimes there are simple answers to what seem to be complex problems and if we ask the right series of questions we can easily get to the heart of a seemingly challenging problem. I also found that this type of problem solving is very collaborative and as such would be very effective when conducting a program evaluation. By involving the stakeholders and intended users in the evaluation program, program evaluation becomes more successful. As presumably many stakeholders are not versed in complex problem solving techniques, the “5 Whys” would allow them to collaborate effectively with one another, while examining potentially challenging issues. I specifically found the link to the Centers for Medicare & Medicaid Services useful as it contains not only a step-by-step outline of the problem solving process, but also has an organizer that I envision in the middle of a table of my students and each student collaborating and interacting together performing problem solving skills. Moreover, using this visual tool in a program evaluation I feel will cause greater interaction and understanding among different stakeholders.

    My question though is how do you know when this method of problem solving is not effective? Is there a number of “Whys” that you would ask before you realized this method needs other aids such as the fishbone organizer you pointed out? What specific types of problems would you not use this for?

    I know answers to the “Whys” ought be factual, so complex moral problems seem to be ‘out’ in terms of use. Am I correct in this reasoning?

    Again thank you very much for this article and the resources involved. I will put it to immediate use in my classroom and in my workplace.

    Best regards,

    Gordon Laffin B.A. Hons. B.Ed. OCT.

    Reply

Leave a Reply

<<

>>

Archives

To top