My name is Kristie A. Thomas, PhD, MSW and I am a recipient of the 2019 AEA/SAGE Early Career Excellence in Teaching Awards. The Teaching of Evaluation Topical Interest Group asked me to reflect on what I learned at the conference and share some tips and tricks I have learned from teaching evaluation to graduate-level, clinical social work students at Simmons University.
My biggest “aha” moment at the conference came from Ronald Heifetz’s plenary on adaptive change. I was already familiar with his framework, but I had never thought about using it to inform my approach to teaching evaluation. As he spoke, I reflected on the challenges I often encounter when teaching evaluation (e.g., student anxiety, resistance, and anger at having to take the course; beliefs that evaluation is unhelpful or even harmful for clients). What I realized from his talk is that addressing these challenges requires an adaptive change framework. In other words, requiring the course exposes them to the material, but without attention to their attitudes and norms about evaluation, it is unlikely that they will exit with a solid grasp of the material or an appreciation for its importance.
I use several strategies to combat these attitudes and norms.
- Convince me! During the first class I ask students to convince me why evaluation is important. They often report back that evaluation helps them to be more accountable to clients and aligns with the social work code of ethics. Bingo! I then show them the following video to highlight how evaluation can directly improve the lives of clients.
- Understand your identity! To get them to think of themselves as evaluators, I have them identify the sources of data that they collect to inform treatment planning (e.g., self-report, administrative records), and then ask them to draw parallels to data collection in program evaluation.
- Demystify evaluation! Finally, it is important to make the process feel real to students and to demystify it as much as possible, so I always share examples from my own evaluation work. For example, I teach the process of measure development by showing them the many stages of the evaluation measures that I have co-authored with my community partners (e.g., MOVERS). They especially enjoy reviewing the full list of initial items, discussing which items they would drop and why, and learning how we settled on the final items. They appreciate that I am honest about the challenges we encountered and mistakes we made along the way.
In summary, good teaching of evaluation, especially in social work education, involves guiding students through a process of uncovering and replacing any negative attitudes and norms about evaluation that they might hold. Doing so helps to ensure that students truly absorb the content and actually apply it.
The American Evaluation Association is celebrating ToE TIG Week with our colleagues in the Teaching of Evaluation Topical Interest Group. The contributions all this week to aea365 come from our ToE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Kristie, I am currently taking a Masters Level course on Programme Inquiry and evaluation, and although I am not at your level, or even a comfortable level with regards to being considered an evaluator, I found myself flipping through articles, but totally drawn to yours. Thinking of your strategies to combat negative attitudes and norms I think is a great idea. We see these negative attitudes in many fields, and your points got me thinking of not only application within evaluation, but also in my current role as a math teacher.
I teach a system of math that is based on modeling and communicating understanding. The evaluation practice always cause the students issues, and the demystifying evaluation really spoke to me, even if it is not evaluation in the sense we are studying and you are referring. I think I can apply your framework to the beginning of the year classes next year and hopefully attempt to break down these negative attitudes students come in with and hopefully set themselves up for success. Thank you for your article!
Greetings Dr. Kristie A Thomas,
My Name is Casey Wells and I am currently a student in Professional Masters of Education program. I am currently enrolled in a course that focuses on Program Inquiry and Evaluation. I came across your blog about evaluation and the focus on an adaptive change framework to inform your teaching caught my attention.
As you shared your thoughts and experiences around teaching evaluation your focus on demystifying the art of evaluation struck a chord with me. Your direction to have students focus on the positives of evaluation, justify why they think evaluation is important and encouraging your students to make personal collections through data collection. Before I began taking my course on Program Inquiry and Evaluation, I have held a narrow view on evaluation strategies as they related to classroom needs. I had relied on organic measures (anecdotal feedback, smiles, student participation) and had not quantified it into a data set.
The video you shared featuring Jacinta was a powerful tool on how important measurement and evaluation can be to systemic changes. The interconnectedness between evaluators and hosting an open process can help both parties learn from each other. If people don’t know why it is important they might not put the necessary effort in to coordinate change
Your article also lead me to a new focus of study on adaptive leadership Adaptive Change is a new teaching and leadership position. The idea that leaders need to continuously learn to reframe problems is currently entering into our school system. How do we evaluate innovation and creativity?
Thank you for sharing your thoughts,
Casey
Greetings Dr. Kristie A Thomas,
My name is Casey Wells, and I am currently a student in a Professional Masters of Education program. One of the courses I am enrolled in focuses on Program Inquiry and Evaluation. I came across your blog about evaluation, and the focus on Ronald Heifetz adaptive change framework to inform your teaching.
As you shared your thoughts and experiences around teaching evaluation, your focus on demystifying the art of evaluation struck a chord with me. I felt that your instruction to have students focus on the positives of the assessment, justify why they think evaluation is essential and encouraging your students to make personal collections through data collection reflected best practices in teaching. Before I began taking my course on Program Inquiry and Evaluation, I relied on organic measures (anecdotal feedback, smiles, student participation) to measure my day-to-day success. I had not created a formal framework to inform my instruction.
The video you shared featuring Jacinta showed what a powerful tool effective evaluation could be. The story does a great job highlighting the importance of accurate measurement to inform systemic changes. If people don’t know why reporting and data tracking are essential, they might not put the necessary effort to coordinate change.
Your article also encouraged me to pursue additional information on adaptive leadership. Today’s leaders must adjust to novel situations on the fly, and they may need to learn new knowledge or skill to solve the problem.
Thank you for sharing your insight,
Casey