Hello there! Saville Kushner, Professor of Public Evaluation at the University of Auckland. I am here to share some reflections on fear and formulas.
I was in a seminar recently, listening to a woman talk about her fear of entering into evaluation. I remembered fears of mine from when I was new to evaluation and vulnerable to its consequences – well, to unemployment.
But what she said brought to mind a change in me.
As I have lapsed into an age of wisdom, I slide into a seasoned confidence. I walk into an evaluation as a veteran actor breezes onto a stage, blasé about the risks which I have minimised with my formulaics. I am even desperate for a moment of chaos that will take me back to my youthful spontaneity. I guess this is what Csikszentmihalyi called ‘flow’ – with no hint of irony.
But as I listened to her I realised that I, too, am afraid when I enter evaluation – genuinely, tangibly. But not for myself, nor because of authority. I fear my failure to respectfully represent those whose lives are under scrutiny. I do feel the burden of that responsibility. At my age of supposed sagacity I am the formula maestro. I can see a person motivated by an inner psychological need and distinguish them from another motivated by belief or by a rational principle….or by habit. I can see when the evaluand is making a complaint about something irrelevant because they cannot articulate their real grievance. I can feel the evaluand’s discomfort, persuaded into talking of things of which they actually have no interest. And I can empathise with someone who speaks of their values, but who needs to promote their interests.
Lesson Learned: These things make me a good evaluation manager. But as much as they might excite Csikszentmihalyi, they do not make me a good evaluator. Because I certainly and casually fall at the first hurdle. I make too easy assumptions about my respondents in their practices to be able to do what I advise in my writing and my supervision: to represent them in their own terms. To listen to the case. To defy my presumption of telescopic insight.
Lesson Learned: Theory of Change evaluators, impact assessors, realist evaluators, those focused on ‘intended users’ – I have reproached them in my writing for placing methodology over lived experience, for elevating theory over what we should be helping our evaluand to do which is theorising. They are unethical in that particular sense. But how different is what I do which is to smother the evaluand with my seasoning? I, too, elevate a methodological principle. All formulas fail people.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.