AEA365 Curator note: Back in January, AEA365 readers asked to read about how evaluators deliver negative findings to clients and stakeholders. This week, we feature 5 articles with four evaluator perspectives on this topic.
A client called after reading the evaluation report and said, “The data are wrong.” My curiosity rose, as well as my blood pressure, and I wanted to understand more.
Hi, I’m Chari Smith with Evaluation into Action. This phone call was a turning point in my understanding of how to shift clients to understanding program evaluation as a learning opportunity. The rest of the conversation goes like this:
Me: “Can you elaborate?”
Client: “Participants say we aren’t communicating with them about the activities. But we are. We cannot send this to our funder.”
My thoughts: They are in fear of losing funding. Emotions are driving their fear. How do I shift them from a state of fear to a state of learning?
Hot Tips:
©2018. Chari Smith. Evaluation into Action. All Rights Reserved.
- Validate their concerns: “I understand this is alarming to you. We will discuss how to use the data and share with the funder.”
- Educate: “The data aren’t wrong, this is what participants said. This means your communication methods with them need to change. Let’s discuss what that can look like. Instead of emails, how about an initial phone call to all nine organizations? Set up a google group so they can discuss as well?
- Collaborate: We worked together to create a one-page improvement plan, highlighting the finding and providing a brief description how communications will change, and then be measured. This was sent along with the full evaluation report to the funder.
Results:
- Funder was happy to see transparency.
- New communication methods worked, participants reported in later survey the felt well-informed and appreciated the change.
- Client was relieved (me too!), and leveraged that experience to secure additional funds by highlighting how they used data to improve their program.
Lessons Learned: Never, ever email a report. Always go through it with them in person first, and then email it after the meeting.
Rad Resources: I am passionate about this topic, it prompted a white paper: Building a Culture of Evaluation. Please let me know about other resources on this topic. Thanks!
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Chari, thank you for the post, especially the validate-educate-collaborate sequence. I might add another step either before or right after validate. I’ve had experiences in which I first needed to clarify the client’s concern. Was the client concerned about the validity of the data, our interpretation of the data, or a recommendation we made based on the data? Usually, their concern was with the interpretation or our recommendation, but establishing that they agreed with the data gave us a point to work from. Thanks again.