Greetings colleagues. My moniker is Michael Quinn Patton and I do independent evaluation consulting under the name Utilization-Focused Evaluation, which just happens also to be the title of my main evaluation book, now in its 4th edition. I am a former AEA president. One of the challenges I’ve faced over the years, as many of us do, is making evaluation user-friendly, especially for non-research clients, stakeholders, and audiences. One approach that has worked well for me is using children’s stories. When people come to a meeting to work with or hear from an external evaluator, they may expect to be bored or spoken down to or frightened, but they don’t expect to be read a children’s story. It can be a great ice-breaker to set the tone for interaction.
Hot Tip: I first opened an evaluation meeting with a children’s story when facilitating a stakeholder involvement session with parents and staff for an early childhood/family education program evaluation. The trick is finding the right story for the group you’re working with and the issues that will need to be dealt with in the evaluation.
Rad Resource: Dr. Seuss stories are especially effective. The four short stories in Sneeches and Other Stories are brief and loaded with evaluation metaphors. “What was I scared of?” is about facing something alien and strange — like evaluation, or an EVALUATOR. “Too Many Daves” is about what happens when you don’t make distinctions and explains why we need to distinguish different types of evaluation. “Zaks” is about what happens when people get stuck in their own perspective and can’t see other points of view or negotiate differences. “Sneeches” is about hierarchies and status, and can be used to open up discussions of cultural, gender, ethic, and other stakeholder differences. I use it to tell the story, metaphorically, of the history of the qualitative-quantitative debate.
Hot Tip: Children’s stories are also great training and classroom materials to open up issues, ground those issues in a larger societal and cultural context, and stimulate creativity. Any children’s fairy tale has evaluation messages and implications.
Rad Resource: In the AEA eLibrary I’ve posted a poetic parody entitled “The Snow White Evaluation,” that opens a book I did years ago (1982) entitled Practical Evaluation (Sage, pp. 11-13.) Download it here http://ow.ly/1BgHk.
Hot Tip: What we do as evaluators can be hard to explain. International evaluator Roger Mirada has written a children’s book in which a father and his daughter interact around what an evaluator does. Eva is distressed because she has trouble on career day at school describing what her dad, an evaluator, does. It’s beautifully illustrated and creatively written. I now give a copy to all my clients and it opens up wonderful and fun dialogue about what evaluation is and what evaluators do.
Rad Resource: Eva the Evaluatorby Roger Miranda. http://evatheevaluator.com/
The American Evaluation Association is celebrating Best of aea365 week. The contributions all this week are reposts of great aea365 blogs from our earlier years. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hello,
I want to thank Sheila Robinson for this post on Mr. Patton’s work entitled Michael Quinn Patton On Using Children’s Stories to Open Evaluation Dialogues available on the American Evaluation Association website at: http://aea365.org/blog/best-of-aea365-week-michael-quinn-patton-on-using-childrens-stories-to-open-up-evaluation-dialogues/ . My name is Kyle Mogavero and I am a student in the PME program at Queen’s University in Kingston, Ontario (Canada).
I was drawn to this article because of the suggestion of using children’s stories to teach an unfamiliar and potentially intimidating topic to a diversified audience. Setting the tone for interaction between stakeholders and the program evaluator is of paramount importance as a means of ensuring utilization of the evaluation and its recommendations. I felt that the fact that the author spoke from their own personal experience made this article significantly more effective. I was able to make a text-to-world connection with the second hot tip because, as a certified teacher in Ontario,I have seen myself how the children’s story by Dr. Seuss entitled The Lorax has been used as a segue for launching a discussion about environmental issues in fifth and sixth grade classrooms. Consequently, I can see how children’s stories would have a similar effect for adults by framing the the context of the discussion about the evaluation. However while the selection of a fitting story can have a positive impact for generating discussion it is easy to see how a selection of an inappropriate children’s story could have the opposite effect and disengage participants. When teaching students to independently select books at their reading level we teach them the Goldilocks strategy for selecting a book at their reading level. The Goldilocks process involves reading the first three pages of a book and holding up one finger for each unknown word; if there are five or more unknown words then the book is too tough, if there are four or less then the book is just right). My question is, is there a formula or specific method for selecting an appropriate story to open discussion about the program evaluation?
Source:
Retrieved 11 August 2016 from: http://www.busykidshappymom.org/choosing-just-right-books-using/
INTRODUCTION TO EVALUATION ACTIVITY
Which of the THREE LITTLE PIGS built the best house?
In the classic tale of the three little pigs, each of the three pigs builds a house as they leave the family home. THE BIG BAD WOLF destroys the houses of the first two pigs, built out of straw and wood respectively, but is unable to destroy the home of the THIRD LITTLE PIG, which was constructed out of bricks. Therefore, this house was the best house, as it was able to withstand the huffs and puffs of the Big Bad Wolf.
The story illustrates a simple evaluation…one criteria was used to present a result. However, in the real world, evaluation is not so simple a procedure. Looking at the criteria below…who built the best house?
• Lowest cost of materials
• Least amount of construction time
• Greatest ability to relocate house to another location
• Greatest use of ‘green’, ecologically friendly materials
• Least amount of danger to people inside if the house collapses
The type of evaluation that a program may use, depends on the goal of the program, and thus should be closely aligned with the program. It can be complicated to create a meaningful evaluation as there are often many factors at play, especially for programs based on people.
Challenge: create a list of criteria as above where the ‘best house’ is the house made of sticks or bricks.
Now going back to the story…what criteria would be important if the goal is to stand up to the huffs and puffs of the BIG BAD WOLF? What criteria would be important if the goal was to avoid the BIG BAD BANKER (i.e. huge construction costs, borrowing of money, a mortgage) ??
Our goal is for child care programs to offer quality programs for children that go above and beyond the legislative requirements.
Work with the people at your table, and make a list of what makes a child care centre GREAT beyond meeting legislation requirements? Brainstorm!!
Aug. 5, 2016
Hello Mr. Patten,
My name is Kathie Tait-Rayner, and I am responding to your article entitled: Using Children’s Stories to Open Up Evaluation Dialogue from The American Evaluator Association’s website (http://aea365.org/blog/?s=children+stories&submit=Go) . I am responding to this article as part of a course through Queens University (Kingston, ON); the course is called Profession Inquiry and Evaluation.
I was very interested in your practise of using children’s stories as a tool for introducing the concept of evaluation. I often use children’s stories in my work with early childhood education students, but hadn’t considered this type of usage before. Children’s stories are often simple and light (on the surface at least), and I can easily see how this tool would be beneficial for a group of people not familiar with evaluation. I have come to appreciate in my class how very complex evaluation is, particularly the evaluation of programs with a goal of changing the way that people think. Evaluation can be much more than an analysis of the end product, and can occur in many different social contexts, all with their own competing areas of emphasis.
I sat down to compose my own children’s story, based on your example of Snow White. I ended up creating a participatory activity that workshop participants could use as they started thinking about evaluation. I plan to use the activity in my volunteer work with a local group of early childhood educators, with a goal of improving the quality of child care.
Thank you for your valuable insight into how to introduce the multi-dimensional work of evaluation. I look forward to using these concepts in my work.
Kathie Tait-Rayner
Queens University
Professional Masters of Education student
INTRODUCTION TO EVALUATION ACTIVITY
Which of the THREE LITTLE PIGS built the best house?
In the classic tale of the three little pigs, each of the three pigs builds a house as they leave the family home. THE BIG BAD WOLF destroys the houses of the first two pigs, built out of straw and wood respectively, but is unable to destroy the home of the THIRD LITTLE PIG, which was constructed out of bricks. Therefore, this house was the best house, as it was able to withstand the huffs and puffs of the Big Bad Wolf.
The story illustrates a simple evaluation…one criteria was used to present a result. However, in the real world, evaluation is not so simple a procedure. Looking at the criteria below…who built the best house?
• Lowest cost of materials
• Least amount of construction time
• Greatest ability to relocate house to another location
• Greatest use of ‘green’, ecologically friendly materials
• Least amount of danger to people inside if the house collapses
The type of evaluation that a program may use, depends on the goal of the program, and thus should be closely aligned with the program. It can be complicated to create a meaningful evaluation as there are often many factors at play, especially for programs based on people.
Challenge: create a list of criteria as above where the ‘best house’ is the house made of sticks or bricks.
Now going back to the story…what criteria would be important if the goal is to stand up to the huffs and puffs of the BIG BAD WOLF? What criteria would be important if the goal was to avoid the BIG BAD BANKER (i.e. huge construction costs, borrowing of money, a mortgage) ??
Our goal is for child care programs to offer quality programs for children that go above and beyond the legislative requirements.
Work with the people at your table, and make a list of what makes a child care centre GREAT beyond meeting legislation requirements? Brainstorm!!
Pingback: Sara Vaca on Creativity and Evaluation Part I · AEA365