I am Mende Davis, co-chair of the Assessment in Higher Education TIG and research assistant professor in psychology at the University of Arizona. I’ve been involved in a variety of higher education evaluations in Minnesota and Arizona.
Student progress in college is usually measured by a series of dichotomous variables; completing 12 credits a semester, passing grades, and on-time graduation. Graduate school follows a similar pattern; admission, coursework, comprehensive exams, theses, and the final degree. Every one of these student outcomes is important, but once each hurdle is passed, it recedes into the distance.
Hot tip: Educational outcomes can be combined into a meaningful scale using Rasch modeling techniques.
The measurement of change is often limited to single outcome variables, even when multiple measures have been collected. This is not limited to Higher Education evaluations; it’s the norm in many fields. Evaluators and administrators collect the data that are required for program reviews. But, we as program evaluators can combine the data that we already have to create a continuous scale. Combining educational outcomes into a scale to measure change can result in greater sensitivity to intervention effects. An educational pipeline scale can incorporate multiple types of indicators, multiple sources of data, and even processes that play out over time.
At the University of Arizona, we successfully piloted a ‘Pipeline’ measure to track progress in graduate school as part of a STEM program evaluation. The pilot was successful and we are using the Pipeline measure in other departments.
Rad Resources
Psychometric software has a great list of Item Response Theory (IRT) computer programs. The Rasch Measurement Analysis Software Directory has an extensive list of computer programs for Rasch modeling. Both pages include commercial and open source software.
The American Evaluation Association is celebrating Assessment in Higher Education (AHE) TIG Week. The contributions all this week to aea365 come from AHE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.