Hi, I’m Steve Powell, a freelance evaluator. I’m really interested in theories of, about, and within evaluation, and the conceptual headaches they can bring with them.
Sometimes as evaluators we are so busy with practical challenges that we don’t have much time to worry if there is sufficient agreement about the meaning of the words we use – but that can lead to a lot of unnecessary arguments. Especially when we use difficult words like “attribution”, “impact” or “intention”, which different evaluation theories might define in different ways.
When doing workshops, I’ve found it useful to present exaggerated versions of these kinds of problems as evaluation paradoxes. Puzzles and paradoxes have often been used in philosophy, from Zeno and Zen to the Sufi mystics, to help us question and sharpen up our understanding of the words we use and demonstrate the importance of reflecting on evaluation theory. Applying different theories might lead to different conceptual responses to the paradoxes.
Below is one such paradox.
An evaluation puzzle: “Billionaire”
A billionaire left 10 million EUR in his will to establish a trust, with instructions that it should be used to “just do good”.
During the ten years since then, support for same-sex marriage has shifted from 10% to 80% public approval, and almost all liberals are now in favour.
In the ninth year, the trust gives 1 million to a campaign for a law on marriage equality, which substantially contributes to the passing of the law in the tenth year.
We don’t know for sure, but most likely the billionaire, like most of his peers and friends, did not support marriage equality when he was alive. But most of his peers and friends now support it.
Now, in the 11th year, you are asked to evaluate whether the trust was used effectively and whether the activities were relevant to the intentions of the billionaire.
If we tried to follow positivistic evaluation principles and retrospectively operationalised “doing good” by providing concrete indicators for it, we would have to decide whether to do this in a way which is acceptable now or which would have been acceptable ten years ago. Whereas if we followed Michael Scriven’s ideas about the logic of valuing, we might even try to argue that there are ways of deciding what “doing good” is which are based on facts and not merely on what we or other people value.
So different evaluation theories give us different ways of deciding what “good” means.
And so, reflecting on evaluation paradoxes can help us to sharpen up our understanding of the words and concepts we use, and help us understand the importance of theory and their implications in practice.
Rad Resource:
I’ve posed a few more evaluation paradoxes here.
If you know of any similar paradoxes, I’d be really interested in hearing about them.
The American Evaluation Association is celebrating Theories of Evaluation TIG week. All posts this week are contributed by members of the TOE Topical Interest Group. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.