My name is Kylie Hutchinson. I am an independent evaluation consultant and trainer with Community Solutions Planning & Evaluation (www.communitysolutions.ca). I give regular webinars and workshops on evaluation topics for both the AEA and CES and Twitter weekly at @EvaluationMaven.
The field of evaluation is rife with inconsistent yet overlapping terminology that stems from different sectors, funders, geography, and individual program areas. Although this terminology is underpinned by consistent concepts and theory, it regularly confuses new evaluators or program staff tasked with doing evaluation in their organization. Despite the fact that more and more funders are demanding increased evaluation activity, this inconsistency in terminology is not going to be harmonized any time soon.
Rad Resource: The free, searchable Evaluation Glossary Mobile App for iPhones and Androids. This app provides immediate reference to over 600 evaluation, program planning, and research-related terms. It is useful for:
- new evaluators
- program staff new to the area of evaluation
- executive directors, program staff, and evaluators needing to interpret inconsistent evaluation demands from multiple project funders
- evaluators working in international contexts
- funders wishing to harmonize their use of evaluation terms.
At some point we also hope to have both a web-based and multi-lingual version as well.
Rad Resource: A Typology of Evaluation Terms. This work-in-progress is my attempt at grouping our overlapping terminology in meaningful ways and perhaps starting more dialogue on this issue.
Hot Tip: Becoming an “interpreter” of different evaluation terminology is a useful skill for any evaluator who works in various program areas and across sectors.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hi Kylie,
Thanks for your helpful post! I’m a Master’s student completing a course on program evaluation, and I can attest to your comment about the struggles faced by ‘newbies’ to this arena! The first few chapters of Chen’s work on evaluation overflow with new terminology and concepts – so much so that I definitely found myself losing sight of the forest for the trees!
The resources you provide are fantastic – what a great and helpful idea to collate program evaluation terminology into one place. Program evaluation is a relatively new discipline, so I suppose that it’s not particularly surprising that concepts and terminology are still developing. But I’m surprised that there’s enough to fill up a glossary of over 600 terms!
Do you think there is work to be done in this field in terms of normalizing meaning so that there are not so many overlapping concepts and terms – often with such seemingly minute differences in definition? I think of the debates between (eg) Patton, Weiss, Kirkhart and others about the appropriateness of descriptors like “use” vs. “utility” vs. “utilization”. Donaldson notes that “a newcomer to evaluation, and even a grizzled veteran, could have a difficult time sorting through the closely related and sometimes interchangeable terms that litter the evaluation landscape” (Donaldson & Lipsey, 2011, p.3)
Is it just a fact of life that evaluators need to become familiar with all these terms and their definitions… or has the evaluation community spent too much time delineating the minutiae?
Thanks!
Dan
References:
Chen, H.-T. (2005). Practical program evaluation: Assessing and improving planning, implementation, and effectiveness. Thousand Oaks, CA: Sage Publications Read: Introduction, Fundamentals for practicing program evaluation, and A practical evaluation taxonomy.
Kirkhart, K. E. (2000). Reconceptualizing evaluation use: An integrated theory of influence. In V. Caracelli and H. Preskill (Eds.), The expanding scope of evaluation use. New Directions for Evaluation, 88 (pp.5-23). San Francisco; Jossey-Bass.
Hi Kylie
Wow! I am so happy to have found this post by you. The website and mobile app that you have shared are incredibly rich with comprehensive tools and information related to evaluation and planning. As someone who is relatively new to program evaluation it is reassuring to hear from experts that, “the field of evaluation is rife with inconsistent yet overlapping terminology” and that it, “regularly confuses new evaluators or program staff tasked with doing evaluation in their organization,” as you so nicely put it.
That said, I am very curious as to why you noted that, “despite the fact that more and more funders are demanding increased evaluation activity, this inconsistency in terminology is not going to be harmonized any time soon” – is this the only future for evaluators?
As another reader commented, it seems that there is only space for further confusion if efforts to reign in and organize the everyday jargon of program evaluation are not drastically and immediately increased. After all, don’t the best forms of communication take place using simply language? Even in complex worlds, simplicity is possible.
Thanks again for sharing this wonderful resources!
Pingback: Evolution of an Evaluator: Kylie Hutchinson | Sheila B Robinson
The app is awesome. Thanks so much for sharing. Great resource. It is downloaded on my phone.
Shana!
I agree with your words. Your app should help.
While people continue to be confused by the terminaolgy in use they will continue to invent their own terminology for their own specific problems and the terminology problem escalates.
I would like to see evaluators using plain english (if that is the language it is written in)- plain english evaluation. New words or new meanings for words indicate to me someone who is either unable to communicate, someone who is trying to sell you something, or someone obfuscating.
I’m loving this app! What a fabulous idea this is! I’ve just added it to my grad course syllabus under “Recommended Apps” (a brand new category, of course), and will introduce it to my students this week. Thanks Kylie!
Such an awesome resource! Thanks for your leadership and innovation, Kylie!