Hello, I am Barb Goldsby, Interim Director of the Exceptional Student Services Unit and Supervisor of the Secondary Transition Services Team from the Colorado Department of Education. We truly value meaningful evaluations of our training, technical assistance and professional development. With the help of the National Secondary Transition Technical Assistance Center (NSTTAC), we have taken Guskey’s evaluation components to heart.
Lesson Learned:
- Participants Don’t Always Know What They Don’t Know. In the area of participant learning (Level 2), we discovered that pre/post evaluations were the best way to measure participants’ learning. However, we discovered that when participants completed the pre evaluation before the content was presented, they had a tendency to rate themselves high because they simply didn’t know what they didn’t know. After the training, they realized what they didn’t know and would rate themselves lower.
Hot Tip:
- Conduct a Post-then-Pre Test. To combat this skewing of data, we developed a pre/post evaluation that participants would complete after the content was given. This way, they could truly think about what their knowledge base was pre-content and post-content. We use a Likert-like scale of 1 to 5 with 1 being a low level of knowledge and skills to 5 being a high level of knowledge and skills.
- Shared Experience and Resources Help Improve Practices. We utilize this evaluation system in all of our trainings and then use the data to inform future trainings. Different teams across our unit are now utilizing this evaluation system in their trainings. Here is an example of the evaluation form we use with participants and the system used to analyze the evaluation data from our trainings.
Rad Resources:
- Tom Guskey’s book Evaluating Professional Development
- NSTTAC Evaluation Toolkit
- Colorado Excel Post-then-Pretest fully functioning spreadsheet
The American Evaluation Association is celebrating the Evaluating Professional Development Community of Practice (PDCoP) Week. The contributions all week come from PDCoP members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluator.
Drawing on the Kirkpatrick Level-2, I am pleased to share the LBAT Model and a case study on it.
The case study is about the experience of the TESP M&E Unit of the evaluation of the trainings of HEIs staff in Pakistan. The case study explains how the M&E Unit developed the Learning Based Assessment of Training (LBAT) Model to analyze the pre-existing learning, the net learning and the learning lag. The Model produced surprisingly interesting results for improved accountability and planning for future training.
For Details, please click on https://www.academia.edu/13100038/The_Level_2_of_Training_Evaluation._The_LBAT_Model_A_Case_of_Training_the_HEIs_Staff_in_Pakistan