I am Vardhani Ratnala and work as a freelance consultant in Monitoring and Evaluation. In this post, I would like share my experience of presenting evaluation findings.
Following an evaluation field visit, the findings are shared with the project management/implementing team. Quite often, when the findings are negative, it ruffles egos and leads to a conflict. Not only are the evaluation findings questioned, but also the methodology, and in some instances the credibility of the evaluation team itself is questioned. As a result, there is a danger that the evaluation findings may not be acted upon and it ends up being a ‘shelf report’.
Lessons Learned:
As an evaluator how do you handle this post-presentation conflict –
1) Cool off – allow the implementing team to raise their objections. Do not react, respond!
2) Ask them to go through the draft evaluation report thoroughly and see if the findings are justified.
3) Set another date for discussion, in which you show them the evidence backing each finding. Following that, request them to divide their responses to each finding into those which are acceptable, partially acceptable and unacceptable.
4) Ensure the language used in the report and in the discussion is positive.
5) Throughout this process, reiterate that the evaluation was a tool for ‘learning’ and not ‘accountability’.
6) For those findings which are unacceptable, include a section under ‘Management Response’ in the final report, explaining the conflict between the evaluation findings and the implementing team’s response. See table below –
1) Ensure their involvement – it is important to involve the implementing team in planning and designing the evaluation. They should be kept informed of the evaluation progress and preliminary findings.
2) Community buy-in – the evaluation team should have a de-briefing session in the field (i.e. with community members and other stakeholders) to ratify the evaluation findings. This would further strengthen their arguments.
3) A spoonful of sugar helps the medicine go down – try to convey a negative finding in positive language. Highlight positives, leading slowly to negatives. Use a ‘praise sandwich’ (sandwiching a negative between two positives).
Tips for Delivering Negative Results
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Good afternoon Sheila,
My name is Jennifer and I came across your article through my Program Evaluation course as I’m currently studying in the Professional Master of Education program at Queen’s University.
Recently, my course has focused on dilemmas in evaluation use so I was interested to read an article on the AEA365 blog that deals with managing conflict following negative evaluation findings.
I find Vardhani Ratnala’s documented experience of delivering negative evaluation findings insightful. Ratnala’s first-hand experience as a freelance evaluation consultant I’m sure provides many opportunities to observe a variety of human reactions and responses when delivering evaluation findings.
Although I do not have professional experiences as a program evaluator, I am not entirely surprised by the reactions of the project management/implementation teams that Ratnala listed. I think that any time individuals are emotionally attached to a program, there is opportunity for evaluation findings to be questioned.
I particularly appreciate the suggestions made in #1 & #5 in “Lessons Learned”. A “cooling off” period is highly advantageous and I agree with it’s placement at the top of this list. Taking time to listen and respond instead of reacting is a great tool to use when there is conflict and tension.
In terms of the suggestion to highlight that the evaluation is for learning is a great idea and might help those involved re-frame their initial reaction to evaluation findings.
The “praise sandwich” idea in the Hot Tips section is one way of delivering findings and makes me think of our approach in education of providing strength-based feedback. Bringing forth and highlighting the positives first, as Ratnala suggests, is a great way to deliver a strength prior to entering into discussions about negative findings.
So much food for thought in this blog post. I’m off to check out the linked Rad Resource – “Tips for Delivering Negative Results”.
Best,
Jennifer
http://aea365.org/blog/vardhani-ratnala-on-assessing-your-d-a-t-e-donor-attitude-towards-evaluation/
A follow-up post on donor attitudes towards negative evaluation findings
Pingback: WE Week: Rodney Hopson and Sonya Horsford on “But can you do it” questions of evaluator credibility and organizational capacity: The nuances of evaluator credibility · AEA365
I agree with this Stan, this is a great and important posting.
One thing I do when I’m anticipating some push-back is craft a few sentences that I want to remember to use in the moment, e.g. “Unfortunately, that’s not what the data is telling us.”, etc. etc.
Kylie
When trying to understand a project’s reaction to negative aspects of an evaluation report, I’ve found it helpful to consider what aspect(s) are in play. Does the project disagree with:
1. the evaluation data collected (i.e., the data themselves are wrong);
2. the evaluator’s interpretation of the data; and/or
3. the evaluator’s recommendation based on the data and interpretations?
Pinpointing the areas of concern has helped us and the project address the concerns without talking past each other.
In my 35 years in the field and 21 years involvement with AEA this was without question an excellent blog that is so true and worthwhile advice for any serious evaluation practitioner in the field.
In one word: Superb!