Hello teacher education colleagues, particularly those charged with leading accreditation efforts for the Council for the Accreditation of Educator Preparation (CAEP). My name is Kevin Eakes and I am the Assistant Dean for the School of Education, Health, & Human Performance at the College of Charleston (CofC) in Charleston, South Carolina. One year ago I started directing CAEP work for CofC’s educator preparation program. As part of the past year’s CAEP journey, I delved into our unit-wide assessments and want to share resources and insights related to validity and involvement of PK-12 partners to address CAEP standard 5.
Evaluators are always keenly concerned with employing valid and reliable instruments. Additionally, an essential component of the quality assurance system for CAEP standard 5 is assessment validity and reliability. The Lawshe Content Validity Ratio (CVR) is a great approach to measure content validity.
The Lawshe method is direct. Experts are provided the instrument and rate each assessment item as either Essential, Useful but Not Essential, or Not Necessary. The greater the number of experts who deem an item Essential, the higher the CVR. The formula to compute the CVR is provided in this 2014 article by Ayre and Scally and can be calculated without the use of statistical software.
Who are experts, how should experts rate items, and how many experts should I include? These are great questions I learned on the job. Experts are professionals who have insight regarding the assessment content. For teacher education assessments, experts may include faculty members, student teaching supervisors, PK-12 cooperating teachers, and PK-12 administrators (both school-based and central-office based). Experts may complete the process in-person or online via a survey or shared document. I have used both formats with equal success.
And, the more qualified experts, the better. The minimum number of experts for the Lawshe CVR is 5. Linked in this article is a table that details the number of experts required for a panel and the number of experts who must rate an item Essential for the CVR to be acceptable.
Outcomes from the Lawshe CVR process provide great insights into the usefulness of assessments. At CofC, we found that two assessments had served their purposes and we created a new assessment based on information gained through CVR work with internal and external partners. Involving external PK-12 partners in content validity not only improved our EPP’s assessments, but also provided opportunities for co-constructed expectations and measurements, key components of CAEP standard 2.
Best wishes on your assessment and accreditation paths; please share experiences and resources!
The American Evaluation Association is celebrating Consortium for Research on Educational Assessment and Teaching (CREATE) week. The contributions all this week to aea365 come from members of CREATE. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to firstname.lastname@example.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.