We are Dayna Albert (Project Coordinator) and Rochelle Zorzi (Editorial Board Co-chair) of the Evaluation Stories Project, an EvalPartners Innovation Challenge recipient. Our project will soon launch an International Call for Evaluation Stories. The purpose is to:
- Identify and share stories of evaluations that have made a difference
- Increase the demand for and use of evaluation
Minimal literature exists on the benefits or impacts of evaluation use, particularly from the perspective of evaluation users. Furthermore, most evaluation literature is very academic. Our project will employ a story-telling format in order to better communicate the benefits of evaluation use to evaluation users.
As an international project, one of our challenges is to reach a multilingual audience despite limited translation resources. A second challenge is to explain what we mean by evaluation impact – a concept that turns evaluative thinking on its head and tends to be misconstrued.
Lessons Learned: Anticipate that people may have difficulty ‘getting’ a new concept. Words alone can be inadequate and ambiguous.
Use story to explain new concepts. Here is a story that Chris Lysy helped us develop to explain the concept of evaluation impacts.
(Click here to see the video!)
Hot Tip:
– Follow-up with clients after an evaluation to reflect on and track evaluation impacts.
– Act now! The Call for Evaluation Stories is a great opportunity to reconnect with a client and explore their interest in participating in the Call for Evaluation Stories
Rad Resource:
– To reach a multilingual online audience, add Google’s Website Translate plug-in to your website. Albeit imperfect, it provides a free and virtually instantaneous website translation.
– To translate a blog, paste the following code into a text widget. Insert your blog’s URL where indicated. The code is written for English (en) to French (fr) translation. For English to Spanish translation, replace ‘fr’ with ‘sp’ and ‘français’ with ‘español’.
<a href=”//translate.google.com/translate?u=http%3A%2F%2FyourblogURL&hl=fr&ie=UTF8&sl=en&tl=fr”” title=””français“”><img src=”http://yourblogURL /2010/02/icons-flag-gb.png” alt=”français” /></a>
Get Involved:
- Join the Evaluation Stories Blog
- Contribute to the discussion on evaluation impacts
Rad Resources: See these posts for additional information on evaluation stories:
- Susan Kistler on VOPEs and the EvalPartners Innovation Challenge
- Chris Lysy on Cartoons and Illustrations as Information Visualization
- Susan Eliot on Evaluation Stories
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hello Ms. Albert & Ms. Zorzi,
I came across your post in connection to program evaluation design. I am currently completing my Master’s of Education at Queen’s University and finishing a course in Program Evaluation Design. As you can see in the link I provided, I had to conduct a PED of a program that I was interested in. The focus groups of my program were First Nation communities across Canada. Although, many communities speak English fluently, there are some that require translators or translation resources to help teach the staff. In addition, the program provides training booklets and resources for the staff to use, and to have a translated version able will help the overall impact of the program. Your “hot tips” are simple but useful in assessing specifically the impact evaluation. I found the video to be quite helpful in distinguishing what evaluation impact is exactly. How exactly do you plan to increase the evaluation use? What connections do you have to other countries or international programs that require evaluation? How would you manage to track the data on a large scale? Who are the stakeholders involved? I look forward to your response.
Kindly,
Katie Still
Dear Dayna and Rochelle,
My name is Sadia Kidwai, and I am pursuing my Masters in Education from Queen’s University, in Canada. I am currently enrolled in an interesting course, called Program Inquiry and Evaluation, through which I have been introduced to the AEA365 Blog, which is a great informative resource. In the blog, I came across your article “Evaluation that Makes a Difference: Stories from Around the World”, and I found the concept to be very intriguing and exciting.
While researching for various assignments in my course, I realized that there is little literature online which talks about the impacts of evaluation use from an end-user’s perspective – a concern which you have highlighted in your article. While there is academic material (not a lot though) on evaluation, I did not find many resources which could serve as real-life examples of good and bad evaluation use. Additionally, most of the evaluation use examples online (which I had access to) either relate to the Healthcare field or are in an Educational setting. I desperately looked for case studies on evaluation use in other fields, and of varying scopes (small and large-scale programs) but came across very little data. For assignments, I had to depend on Primers (for Healthcare industry), or academic material to understand evaluation concepts in depth. To make new information meaningful, it is imperative that we see it in action, in a variety of contexts. That’s why I absolutely love the idea of the project you have proposed in your article, where stories of evaluations that made a difference will be shared on an online platform to make evaluation more understandable to the clients. The icing on the cake is that the stories will be collected from around the world, so people like me can see evaluation use in action, relating to various fields and contexts.
I understand that the project has been launched, and I was able to visit your website https://evaluationstories.wordpress.com/ , where I read inspiring Evaluation stories from Kenya and Sri Lanka. I think you guys have done a great job in creating an online location which can be accessed by a variety of people to read exciting stories and case studies on Evaluation. This will help readers understand evaluation use in detail. I found your website to be an amazing resource. Thank you!
Sadia
Ms. Albert and Ms. Zorzi, I am so pleased to have come across your blog entry about evaluation impact and story telling. Your initiative is a great one since there really is nothing more satisfying than seeing the positive impact various policies and programs can have on their intended benefactors. Everyone loves a good story after all and I for one will visit your site on a regular basis.
I am currently pursuing my M.Ed at Queens University in Kingston, Ontario, Canada and am in the midst of studying program inquiry and evaluation. My own program of inquiry is the letters of accommodation program at Niagara College, where I teach as a faculty member. This program is intended to remove barriers of learning for students with physical or learning disabilities by providing access to note takers, extra time for exams or study, advance notice of assignments or other evaluations and specialized computer software designed to help them with reading, note-taking or listening. It is an excellent and important program and I feel confident that by the time I have completed my evaluation of the LOA program that all stakeholders will have been given a voice, that my research will hold rigor and that my recommendations could indeed help to make positive changes in the program going forward. Some of my recommendations might even get used but what real impact will my efforts have on the students who utilize the program and how could I possibly measure these impacts? This data is very difficult to measure considering the sheer number of students using the program (in a multitude of degrees) and the endless possible reasons for success or failure in school. As mentioned, the ultimate goal is to eliminate all controllable barriers to learning in the college environment thus giving our students the maximum opportunity to reach their academic best but how do we measure this and relate it to changes in the LOA program itself? Perhaps simply asking students to share their thoughts on the program after using it and the impact it’s had on their confidence, engagement and academic standing is enough for a good story. I’d be interested to hear your thoughts on this. Thank you.
You have hit on an important issue, for the evaluation being done is any of it actually being used? If it is used is it only considered useful to a client if it just reinforces a prior held opinion or position? Is it necessary to evaluate the evaluations?
That said I agree with your idea of using stories to make an evaluation more understandable to the client. I have read a couple of articles in the popular press that repeat the concept that humans are story telling creatures, that we create narratives to understand and manage the outside world. Evidence that this is true, I believe is visible in presidential politcs where the politician that presents voters with the most appealing story/narrative will win the white House.
Great post, Dayna and Rochelle! I would like to add that we are also in Twitter (@EvalStories), Facebook (https://www.facebook.com/evalstories), and Google Plus (http://goo.gl/RQ5Op0).