Hello, my name is Trilby Smith and I am the Director of Learning and Evaluation at Vancouver Foundation www.vancouverfoundation.ca . We are Canada’s largest community foundation, and in my role I support grantees, foundation staff, community advisors and other stakeholders in learning principles and practices that advance knowledge and understanding of our work. I have been an evaluation practitioner for almost 20 years, and in that time, there has been one consistent theme in my work, relationships. The majority of my career has been spent practicing evaluation (and learning) alongside the people who are most affected by the program/project/initiative being evaluated.
Lesson Learned: Be clear about the purpose of your evaluation. Is it for accountability? Is it for learning? Is it for both? This purpose should drive the methods you choose, and the extent to which you invest in relationship development. Research shows that learning deepens in the presence of relationships. When we are trying to learn from what we are doing, we need valid and truthful information. The validity of your data will be increased if it is collected in the presence of strong, healthy relationships.
Rad resource: I refer people to Better Evaluation all of the time! This page is a great guide to thinking through the purpose of your evaluation http://www.betterevaluation.org/en/plan/frame/decide_purpose
Lesson Learned: Redefine who is an expert. As evaluators, we are most often not the expert in the context that we are working in, we are expert in evaluation. The experts are those with lived experience of the issue addressed by the program/project/initiative you are evaluating. In order to engage those with lived experience in the evaluation and learning work that you are doing, you need to build relationships with them.
Rad Resources:
Here is one of my favourite blog posts about the idea of experts from Katelyn Mack at FSG
Here are some great dialogue-based, democratic tools for engaging everyone in the room on an equal footing
And if you want to read more about our work of building relationships and engaging the experts you can read here
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators
Pingback: My personal year of learning about evaluation – Chris Corrigan
Hello Trilby
My name is Tracy Dietrich; I am a college professor in Ontario as well as a graduate student with Queens University currently completing a course on program inquiry and evaluation. One of the assignments in our current module is to connect with a member of the evaluation community by responding to an article posted on the AEA365 blog.
In your post you refer to the importance of creating relationships with the people who are most affected by the program being evaluated. Last year in my professional life I had the opportunity to be involved in the evaluation of the program I teach, which is the Emergency Telecommunications Program. This program is designed to prepare students for jobs as 9-1-1 operators/dispatchers. The evaluator assigned to our review had very little knowledge of the field, and as a result she and I worked very closely on designing focus group questions with language our community partners and other program users would easily recognize.
Your lesson learned of “redefine who is an expert” resonated strongly with me as a result of the above experience. Had we not used terminology that was recognizable to our stakeholders and other users, the evaluation itself would have been merely symbolic, just another item crossed off the checklist. By the evaluator building a close relationship with myself as well as developing the relationship with the program users, many of the people involved were visibly invested in improving the program and creating recommendations for improvement of achieving the desired outcomes.
Also, thank you for the rad resources listed in your post. The Better Evaluation website is very informative and I am looking forward to trying some of the engagement tools described in the Art of Hosting link.
Sincerely,
Tracy Dietrich