Susan Eliot on Evaluation Stories

I’m  Susan Eliot, an independent qualitative research and evaluation consultant in Portland Oregon. As an applied researcher, I’m continuously searching for creative yet rigorous qualitative approaches that fit my clients’ budgets and capabilities.

I recently employed a method called the evaluation story to elicit feedback from elementary and middle school students enrolled in an AmeriCorps mentoring program. Good stories are captivating and enduring. They are part of every culture and help us make sense of the world. Storytelling can also be a powerful evaluation tool.

Richard Krueger defines an evaluation story as “a brief narrative account of someone’s experience with a program, event, or activity that is collected using sound research methods.” When used as part of a traditional mixed-method evaluation, Krueger contends we can use stories to help programs and organizations make sense of their programs.

Five key factors differentiate an evaluation story from other stories. According to Krueger, an evaluation story:

1. Is a deliberate, planned effort using systematic procedures.
2. Identifies the source of each story.
3. Verifies stories with the storyteller or others familiar with the story.
4. Includes a description of how stories were captured/handled using accepted research protocol.
5. Includes a statement by the evaluator about the degree to which the story represents other individuals with similar circumstances.

Krueger offers the following tips for collecting evaluation stories:

Hot Tip: Using stories to collect evaluation data often takes more time than anticipated. Plan carefully, choose storytellers thoughtfully and engage others in looking out for potential stories.

Hot Tip: To prevent stories from being dismissed as mere anecdotes, describe the process used in obtaining the story, handling the story, verifying the story, and ascertaining its representativeness and meaning.

Hot Tip: It takes skill to get stories out of people. Because it “locks up” people’s memory, avoid asking for a story. Instead ask people to “tell me about . . .”

Hot Tip: Stories usually don’t emerge in perfect form. Expect to invest time in several cycles of editing until the story clearly emerges.

The evaluation story is a great tool for connecting to a program or organization on a personal level. Everyone loves a good story. The trick as an evaluator is to tell a story that resonates with the audience and yet, is grounded in rigorous qualitative methodology. Is it a challenge? It sure is. But the benefits of influencing policy and program development with a well-executed story make it a worthwhile and powerful tool to add to the evaluator’s toolbox.

Rad Resource: Read more about Richard Krueger’s storytelling advice for evaluators on his website, as well as in his chapter on evaluation stories in the Handbook of Practical Program Evaluation.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.