We are Nora Murphy and Keith Miller with TerraLuna Collaborative, an evaluation cooperative in the Twin Cities, Minnesota. We feel fortunate to be the evaluation partners on several large developmental evaluations.
One project we are working on seeks to support the inner wellbeing journey of seasoned social entrepreneurs. On a recent conference call, a project team member asked: “How do you know when to use the data to make a change to the program? Isn’t struggle an important part of the individual’s wellbeing journey? If we react too quickly to data and ‘fix’ everything the participant isn’t comfortable with, aren’t we minimizing their opportunities for growth?”
He’s right. I (Nora) shared my perspective that evaluation data is only one source of information that should be used when making a decision. Also important to consider is: 1) our intuition, 2) our accumulated personal and professional wisdom, and 3) the collective wisdom of the group of people seeking to use the evaluation findings.
Hot Tip: Be reflective and identify the source(s) of wisdom you are drawing on.
Reflecting on that conversation, Keith and I realized that my response was rooted in the guiding principles of a three-year partnership with the Minnesota Humanities Center, Omaha Public Schools, and The Sherwood Foundation. The guiding principles are:
- Build and strengthen relationships;
- Recognize the power of story and the danger of absence;
- Learn from and with multiple voices; and
- Amplify community solutions for change.
These principles guide how we show up as evaluators and how we do our work. Evaluation use happens when there is a foundation of trust–trust in both the results and the evaluators. We’ve learned to build trust by investing in relationships, intentionally including multiple voices, seeking absent narratives, and amplifying community ideas and solutions.
Hot Tip: Be responsive, not reactive.
Patton (2010) suggests that one role of developmental evaluators is to look for and document “forks in the road that move the program in new directions. (p. 150)” As developmental evaluators we can facilitate conversations about whether the data be used immediately because it indicates a fork in the road, or whether the data is something to be aware of and track. During these conversations we can also create space for intuition and wisdom.
Lesson Learned: These guiding principles have helped us shape our role as evaluation partners and increase evaluation use. Our partners trust us to engage them in reflective conversations about what the findings mean how they might be used.
Rad Resource: Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use, Michael Quinn Patton, Guilford (2010).
Rad Resource: Nora F. Murphy and Jennifer Tonko – How Do You Understand the Impact of the Humanities?
The American Evaluation Association is celebrating Evaluation Use (Eval Use) Topical Interest Group Week. The contributions all this week to aea365 come from our Eval Use TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hello,
I came across your blog as part of an assignment for a Queens University Course in Evaluation. What stood out for me in your article is the importance of struggle within a developmental context. As a Learning and Development Facilitator where we train new material to both newly hired staff as well as seasoned employees, struggle to learn new information is often part of the learning journey for my participants. However, it is also a necessary part of development to learn new skills and information. Due to this struggle, we may receive feedback through surveys that may yield some element of dissatisfaction with the training. I do believe that when evaluators of the program look at such data that it is definitely important to “facilitate conversations about whether the data be used immediately because it indicates a fork in the road, or whether the data is something to be aware of and track” in order to ensure data findings are being used effectively.
One crucial element that you mentioned that this depends on is the stakeholders and communities trusting the evaluator to guide them in determining when it is in fact a “fork in the road” or something to simply record and have an awareness of. A lack of trust could therefore lead to a decrease in the use of evaluation because stakeholders may not trust the evaluator’s guidance and findings. You mentioned useful guiding principles as a way of building trusting relationships with stakeholders and communities in order to yield use of evaluation findings. As an evaluator, have you encountered a lack of trust from stakeholders, despite using the guiding principles? How would you overcome that challenge if it were to arise?
Thanks,
Su Lyn
I came across your article recently for a course that I am taking at Queens University called Program Inquiry and Evaluation. What stood out in the article for me was the decision an evaluator should make in regards to either fixing all the problems in a program or working step by step with the participants. I am an Early Childhood Educator and I have found that when a child is having a difficult time with a task, they will benefit more if I step back and allow them to problem solve. I can be their as a guide and mentor; however, they will not learn new skills if I am always running to the rescue.
The guiding principle of building and strengthen relationships as an evaluator was very interesting. In previous readings on program evaluation, there is important data on how to perform an evaluation; however, I felt like there was a piece missing. The missing piece being, “how to develop a trusting relationship where the participants will trust you ‘evaluator’ and the results found?”
It is very important to make sure that participants feel like they are having their voices heard. If a person feels left out of the process, they are likely to be less willing to accept what is being said. In the article it is mentions that your partners trust you to engage with them in reflective conversations; have you every come across a situation where people are less forthcoming even when you try to develop open communication during an evaluation process?
Thank you for your time,
Shannen Wright