Hi, my name is Rick Axelson. I am a faculty consultant in the Office of Consultation and Research in Medical Education (OCRME) at the University of Iowa. OCRME works with faculty, staff, and students to support innovation and improvement in medical education practices. My colleague, Susan Lenoch, and I conduct workshops and individual consultations with clients – primarily educators – on program evaluation. The workshops attract faculty and staff from across the College of Medicine, all with different needs and interests.
One of the biggest challenges we’ve encountered in designing workshops for audiences of non-evaluators is “where to start.” Participants’ background knowledge seems to vary greatly and, there isn’t sufficient interest, time, or commonality in background on which to base our discussion. So, where do we begin?
Hot Tip: Start at the end. That is, build backwards from the participants’ common goals or interests. Why are they interested in doing an evaluation? Is there a particular product that they need to develop? To provide a common interest for participants, we advertise our evaluation workshops as an aid for completing the evaluation portion of an in-house grant application. Participants are encouraged to bring their grant proposal ideas to the workshop; we open the workshop by discussing their project ideas and refer to their projects throughout the workshop to illustrate evaluation principles. The added advantage of this approach is that it also works as a warm-up activity.
A second major challenge with these workshops is to provide a simple, straightforward process that will enable participants to create their desired evaluation product. Here, we have found the following to be helpful.
Hot Tip: Use examples of completed products to illustrate the evaluation design process. We walk participants through an example of a previously funded project. Breaking the evaluation design process down into a few simple steps, we show how the sample evaluation would have been developed using this method. We also point out areas (e.g., statistical analysis) where participants may need expert assistance to complete their study design.
Rad Resources: We have developed a workbook to lead participants through the development of the evaluation section for an in-house grant application. A previously funded project was used as the central example throughout the workbook to illustrate the steps involved in designing a program evaluation for an educational intervention. You can download it from: http://www.healthcare.uiowa.edu/ocrme/teach_train_sup/online_lean_teach_edu.htm
This aea365 Tip-a-Day contribution comes from the American Evaluation Association. If you want to learn more from Rick on this topic, check out the sessions sponsored by the Teaching of Evaluation TIG on the program for Evaluation 2010, November 10-13 in San Antonio.