GSNE Week: Laura Pryor on Considerations for Teacher Evaluations with Multiple Measures

Greetings, I am Laura Pryor. In addition to being a GEDI alumna, I am a student at UC Berkeley’s Graduate School of Education in the Quantitative Methods and Evaluation program. As part of my graduate evaluation work, I have been exploring the recent trend of using multiple measures to evaluate teachers. As part of this trend, many policymakers and district leaders are combining multiple measures into a summative composite score, often for the purposes of high-stakes decision making (such as salary and personnel).

As a graduate student evaluator, I have been exploring two questions:
1) Is it necessary and/or purposeful to create a composite score?
2) If so, how should an evaluator combine multiple measures into a single composite score?

I hope this post provides insight into these questions so that evaluators can more easily navigate the increasingly popular context of high-stakes teacher evaluations.

Hot Tip 1: The purpose of the evaluation should decide if a composite score is needed. While it may be a current trend, not all multiple measure evaluation systems are used for a personnel or salary decision. For many districts and schools, the evaluation system is used to help teachers/staff identify areas for improvement; in this case, a composite score is not always necessary. If the evaluation system is intended for multiple purposes, prioritize purposes with stakeholders and discuss the feasibility for the evaluation system to embody multiple uses.

Hot Tip 2: If creating a composite score, select a model that is most appropriate for the evaluation:
a. The conjunctive approach: A pass/fail score is given; individuals must score at a specified passing level on every measure.
b. The disjunctive approach: A pass/fail score is given; individuals are only required to score at a passing level on one of the measures.
c. The compensatory approach: Individuals are given a continuum of scores; low scores on certain measures are compensated for by high scores on other measures.

Hot Tip 3: When using a compensatory approach, decide how to combine the measures:
a. Clinically: Evaluation stakeholders decide how to weight each measure; this is often called the ‘eyeballing’ approach.
b. Statistically: Select a criterion target and use regression methods to statistically determine the weights for each measure; this approach is considered more accurate than the clinical approach.

Rad Resources:

AEA is celebrating GSNE Week with our colleagues in the Graduate Student and New Evaluators AEA Topical Interest Group. The contributions all this week to aea365 come from our GSNE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.