Hi! I’m Amy M. Gullickson, 2010 IDPE graduate and Acting Co-Director and Senior Lecturer, University of Melbourne Centre for Program Evaluation, Melbourne, VIC, Australia.
One of the challenges to graduate education in evaluation is the Dunning-Kruger effect. Because evaluation is an everyday activity and a professional activity, and all the other kinds of activities in between, anyone can be asked to do evaluation and believe they know enough to get the job done. While some organizations are credentialing evaluators, that is not common practice across sectors and disciplines. When the different levels of knowledge, skill and proficiency across novice to expert aren’t described, it’s understandable that anyone can assume they are an expert – they don’t know what they don’t know. That’s our Dunning-Kruger problem.
So why be a person who pursues an advanced degree in evaluation? I’ll give you a snapshot of reasons based on my experience with the IDPE.
Lessons Learned:
Resources: The history and literature base of evaluation is broad and deep. If I hadn’t had a university library and graduate level course work, I might never have discovered what I didn’t know, and how much of it there was. I also might not have had anyone to teach me how to search it and use the literature. Access to university libraries and university librarians is one of the life-long gifts of a graduate level education.
Thinking: Studying with Michael Scriven gave me a grasp on the foundational logic of evaluation, its tasks and requirements. I realised evaluation is multi-disciplinary (requiring expertise from a variety of disciplines) and transdisciplinary (a set of skills, knowledge and working logic that all the disciplines need to function effectively). Without that experience, I would not have understood evaluation as a distinct transdiscipline focused on generating evaluative judgements that transparently bring together justified values and credible evidence. That frames my teaching, research and practice, our fully online Masters and Graduate Certificate program at The University of Melbourne, and my ongoing supervision of PhD students.
Experiences: I moved from junior roles to senior leadership on local, national and international evaluation projects over the course of my 7 years. My experiences gave me the confidence and knowledge to launch and run my own evaluation consultancy, and now serve as an Acting Co-Director of an international evaluation center – delivering high quality processes and products to clients like the National Science Foundation.
People: International experts worked at and just stopped by at WMU. My classmates were from disciplines ranging from dentistry to nuclear science. I made lifelong friends and colleagues that I still call on when I need a critical review or a collaborator – because I know they will ask me the hard questions I need to answer to improve my thinking and work.
Graduate education in evaluation provided me i) a more accurate picture of evaluation expertise, ii) foundational space and experiences that enable me to continually adapt, learn and grow towards it, and iii) positioned me to be able create the space and experiences for others to develop theirs.
This week, we’re diving into learning about the Interdisciplinary PhD in Evaluation (IDPE) program at Western Michigan University. IDPE is the oldest evaluation doctoral program whose purposes include the education and development of thought leaders in evaluation. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hello Amy
Thanks for writing this post. I am taking my Professional Masters in Education at Queens University in Canada. One of the courses I am taking now is about evaluation. If I am being honest with you, I thought I knew something about evaluation but as the course has been going along and I have doing the readings, watching the videos, doing assignments, and discussing different ideas with my classmates I have discovered that the field is so vast and there is so much to learn even after the class is finished.
I enjoyed your reflection on the lessons that you have learned. Like you the more I have studied the more I have discovered that I do not know much. However, I have not dedicated my higher learning to this topic and would have better knowledge than I would. I have been introduced to Chen and his book Practical Program Evaluation which is the main text for our course. This book has provided me with a framework to understand the different moving parts to evaluation. Using the Dunning-Kruger effect you have demonstrated with more knowledge you obtain the more your grown you understand what evolution thought education, experience, and connecting with experts.
I would like to know your views on evolution at the beginning compared to now. I am also curious to see if there are any differences in theory since we live in two different countries. I also wonder from the international experts you have worked with have you worked with people from Canada
I was also not aware of the Dunning-Kruger effect and am glad that I was introduced to it. After watching the video, I can see how you can not only apply this to evaluation but other professions as well. When I started teaching right out of university 11 years ago, I thought I had all the answers. But when I started to teach full time, I realized that I only knew the theory behind teaching and very little about the practice. I had confidence and at that point not a vast amount of knowledge. I had to admit to myself that I did not know much and set myself on a path to learn more. I found the more I learned to more I did not know. I was open to listening to colleagues, taking part in professional development, and taking courses to expand my knowledge. I have witnessed a new teacher starting at our school a year ago and she has the Dunning-Kruger effect. She believes she has all the answers and will not take any advice from other staff who have been teaching for many years and has given others the impression she is the best teacher the school has since she is fresh out of university. She does not realize yet of all the stuff she does not know but she has an unbelievable supply of confidence in her abilities.
Thanks for sharing your ideas.
Sarah
Amy,
Your post made me think about a quote attributed to many so I grabbed this one from Lorenz. “Philosophers are people who know less and less about more and more, until they know nothing about everything. Scientists are people who know more and more about less and less, until they know everything about nothing.” ? Konrad Lorenz
Thanks for your posting in AEA365.
Tom
I wonder what the evaluator equivalent would be… evaluators are people who…
I’ll mull that over. :^) Thanks for the provocation!