Hello! We are Dana Linnell Wanzer, evaluation doctoral student, and Tiffany Berry, research associate professor, from the Youth Development Evaluation Lab at Claremont Graduate University. Today we are going to discuss the importance of high quality relationships with practitioners in evaluations.
“In the absence of strong relationships and trust, partnerships usually fail.”
-Henrick, Cobb, Penuel, Jackson, & Clark, 2017, p. 5
Research on factors that promote evaluation use often include stakeholder involvement as a key component (Alkin & King, 2017; Johnson et al., 2009). However, collaborations with practitioners are insufficient to promote use; rather, partners must also develop and maintain high quality relationships. For example, district leaders stress the importance of building productive relationships for promoting use of evaluations in their district (e.g., Harrison et al., 2017; Honig et al., 2017).
The importance of high quality relationships has been stressed through the focus on participatory or collaborative approaches to evaluation and through the inclusion of interpersonal factors in the evaluator competencies. Furthermore, utilization-focused evaluation (Patton, 2008) states that “evaluators need skills in building relationships, facilitating groups, managing conflict, walking political tightropes, and effective interpersonal communications” (p. 83) to promote use.
Lesson Learned: In our experiences as evaluators, the programs that have made the greatest strides in using evidence to inform decision-making are those who have a strong, caring relationship with the evaluation team. We genuinely want to see each other succeed; we are friendly and enjoy being together. We do not approach the relationship as a series of tasks to perform, but rather the relationship affords us the opportunity to dialogue honestly about the strengths, weaknesses, or gaps in programming that should be addressed. Without authentically enjoying each other’s’ company, it becomes a chore to meet and reduces the informal opportunities to chat about using evidence to improve programs.
Hot Tip: High quality relationships are characterized by factors such as:
- Trust
- Respect
- Dependability
- Warmth
- Psychological safety
- Long-term commitment to mutual goals
- Liking one another and feeling close to each other
Rad Resource: King and Stevahn (2013) describe interactive evaluation practice as “the intentional act of engaging people in making decisions, taking action, and reflecting while conducting an evaluation study” (p. 14). They describe six principles for interactive evaluation practice: (1) get personal, (2) structure interaction, (3) examine context, (4) consider politics, (5) expect conflict, and (6) respect culture. They also provide 13 interactive strategies that can be used to promote positive interdependence among partners.
Rad Resource: Are you interested in assessing the effectiveness of your collaboration, especially its relationship quality? Check out the Collaboration Assessment Tool, especially the membership subscale!
The American Evaluation Association is celebrating Ed Eval TIG Week with our colleagues in the PreK-12 Educational Evaluation Topical Interest Group. The contributions all this week to aea365 come from our Ed Eval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Dana and Tiffany,
Like a few others, I’m currently enrolled in a Master’s program at Queen’s University and found your article particularly interesting. The more I learn about evaluation use, the more intrigued I am by how important evaluator/participant relationships are. I recently read a few articles that discussed the role of the evaluator and I’ve been leaning towards agreeing with Patton, who “advocated an active role for evaluators in promoting and cultivating use” (Shula, 1997). No doubt an evaluator ought to strive for their evaluation to be put to use the way they intended it, but it is a challenging task to forge the types of relationships that would have most/all key players in agreement on how best to make use of the evaluation.
While participatory activities and ongoing stakeholder engagement are obviously crucial to developing the previously-discussed relationships, I imagine that the first few days are the most important for setting the tone of the evaluation and creating the right type of environment.
In the professional world, facilitators, developers, and participants can often be quite tied to their respective programs, and I completely understand that feeling. If a certain program were my brainchild, I might be rather protective of any outsider potentially making value judgement and assessments about something that I’m not convinced they will completely understand. Weiss made note of how, even hearing about the news of an upcoming or ongoing evaluation can impact stakeholders/participants due to the fact that the evaluation itself can either legitimize a program or make people questions it’s worth, “depending on their orientation” (Weiss, 1998). This leads me to believe that the most important time in a program evaluation is the first few days when they have their one (and likely only) opportunity to create a positive atmosphere around the process and to lay the foundation for strong, healthy relationships.
During that time, I imagine a major goal for an evaluator is to make those stakeholders (facilitators, developers, participants) feel like their input is not only valued, but that it is imperative to the success of the evaluation. Without having those key members being actively involved in the process, evaluators can’t expect to change anyone’s orientation towards the evaluation or the program itself. I wonder if you have any advice for those few days when an evaluation is in itself infancy stages and opinions on it are still being formed.
Thank you very much for your time!
Colby
Thanks for sharing your thoughts, Colby! A few of mine in regards to your question: Take the time to listen to them and get to know them. Find out what they care about, what their program is doing, and how you can help them. Find out what hesitations they may have towards evaluation, and tell them you are there are determine how well it works, but also to help them improve the program and learn from their program. Show them how much you care about them, their program, and their participants. Make them feel like you are one of their team, whom they can trust and confide in.
Hi Dana and Tiffany,
Thank you for your interesting post! I found it to be straightforward, practical and insightful with connections to your own experiences as evaluators in the ‘lesson learned’ component. The Hot Tip and Rad Resources you offered are great resources that I plan to utilize as I am currently completing a course at Queen’s University in Program Inquiry and Evaluation, with a focus, at the moment, on evaluation use.
This week, focused on dilemmas in evaluation use, I explored dilemmas associated with collaborative approached to evaluation. In your post, you made a statement that “however, collaborations with practitioners are insufficient to promote use; rather, partners must also develop and maintain high quality relationships” and followed shortly after with the notion that “furthermore, utilization-focused evaluation (Patton, 2008) states that “evaluators need skills in building relationships, facilitating groups, managing conflict, walking political tightropes, and effective interpersonal communications” (p. 83) to promote use. While I wholeheartedly agree with these statements, I wonder if you have any ‘Hot Tips’ or advice or experiences to offer on troubleshooting to managing conflicts when all stakeholders may not be on the same page?
While my experience in evaluation is limited, I appreciate your article, particularly the connections to your own experiences as I seek to learn more about collaborative evaluation use.
Thanks again,
Stephanie
Tough question when I think conflicts are so situational! I think it’s important to clarify what each person’s position is; too often we may assume we know the other person’s position but we really don’t. Try to put yourself in their shoes: why do they have the position they have? And then ask yourself (and ask them) what would be wrong about taking the opposite position? Meanwhile, keep your ethics in check and continuously ask yourself if you are upholding them to the best of your ability: is retaining your position or taking theirs going to compromise your ethics in any regard?
Thanks Dana,
I appreciate your feedback and I will look into Interactive Evaluation Practice to see if I can find some more information about this.
Thanks again for taking the time to look over my post!
Josh
Hi Dana and Tiffany,
I would first like to say that I really enjoyed reading this post. I am in the process of completing a course at Queen’s University and I couldn’t help but notice that there is a lot of relevance regarding your post and the recent readings that I have completed.
A recent concept I have been looking at is ‘Evaluation Use’. I find it interesting how evaluation has changed over the years from an accountability approach to a multifaceted entity that involves collaboration between evaluators and stakeholders.
In your article you made a statement that I agree with:
“collaborations with practitioners are insufficient to promote use; rather, partners must also develop and maintain high quality relationships.”
Later in your post you connect back with this idea of relationships by saying:
“In our experiences as evaluators, the programs that have made the greatest strides in using evidence to inform decision-making are those who have a strong, caring relationship with the evaluation team.”
I am not an experienced evaluator, but I am trying to learn more about ‘evaluation use’ and the relationships between stakeholders and evaluators.
After reading your hot tip regarding what characterizes a high quality relationship I started wondering if you have any clear strategies on how to manage relationships that go sour between stakeholders and evaluators?
I think at this point in time I understand what goes into fostering a strong relationship, but do you have any experience where personality or trust issues (for example) had to be mediated in an evaluation? Did any strategy work best to resolve them?
I assume that sometimes during an evaluation all the appropriate effort has been made to cultivate strong collaborative relationships, but sometimes unforeseen issues may still arise. Almost every school department that I have worked in has had some personality and direction issues. Measure were taken such as shifting personnel and having face-to-face discussions, but even to this day these individuals can’t really work well together. I think some alternative strategies would be very helpful.
Thank you very much for taking the time to read my comment.
All the best in the meantime,
Josh
Hi Josh,
Thanks for your interest in our blog post! Your question is a great one that I’m afraid I do not have the answer to. Most of the literature I am finding on the topic discusses building relationships and trust from nothing, not trying to turn around bad relationships that lack trust. However, Interactive Evaluation Practice (Stevahn and King) might have something to say about this topic. I wish you luck; dealing with mistrust is a difficult situation that many are unable to overcome.
I will reach out to Tiffany Berry and see if she has any recommendations!
-Dana