Hello! My name is Alex (Chi-Keung) Chan and I am currently a Senior Lecturer at the Hong Kong Shue Yan University and a partner of A Data-Driven Consulting Group, LLP based in Minnesota. Prior to that, I was a Senior Evaluation and Research Fellow at the Minneapolis Public Schools. I have been adopting the value-added evaluation approach to investigate the effectiveness of various social and educational programs or to identify the beat-the-odds teachers or schools. Unfortunately, stakeholders always emphasize the summative interpretation of the value-added results without fully understanding the potential formative uses of these measures. So how can we add value to value-added to make it more meaningful and useful?
Hot Tips:
1. Choose value-added measures that are sensitive to the implementation process (e.g. improving teaching) but not just for the accountability outcomes (e.g. determining performance pay). In other words, a value-added measure is meaningless if it cannot tell whether what a teacher does is really adding value to the learning of student.
2. Link the value-added findings to implementation (e.g. link the value-added scores with the instructional observation scores) to identify best practices that we can learn from (what works) and practices that need to be improved (what doesn’t work).
3. Gather additional evidence to verify the value-added X implementation interaction results (e.g. interview the principals, coaches, and teachers who are part of the professional development program). Thus, we want to find out the details of what works and what does not work beyond the numbers.
Lessons Learned:
1. Collaborate and select value-added measures with the stakeholders. Sometimes, the opinions of stakeholders could be either too strong or too loose at this initial but critical stage.
2. Communicate and emphasize the goals and benefits of the formative uses of value-added measures to stakeholders. This is a very challenging process because our focus is too often driven by the limited goal of “accountability” by sacrificing the ultimate goal of “improvement”.
3. Encourage and understand the different perspectives of various stakeholders in interpreting the findings. We often miss some voices or ignore some perspectives due to the uneven power structure in a system. We need to ensure that there is a process in place to hear the voices of powerless people who usually are impacted the most by a program.
Rad Resources:
- Value-Added Research Center (VARC) at University of Wisconsin: (Click on the Tutorials tab)
- Value-added Measures in Education by Professor Douglas Harris: (An easy-to-read book for the layperson to understand the concepts and applications of value-added)
Hot Tip: Minneapolis/Saint Paul Activities:
My family loves to have a walk at the Sculpture Garden when we were living in Minnesota, and we love the Quang Vietnamese Restaurant at Nicollet Mall.
The American Evaluation Association is celebrating Minnesota Evaluation Association (MN EA) Affiliate Week with our colleagues in the MNEA AEA Affiliate. The contributions all this week to aea365 come from our MNEA members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.