Hello, my name is Kim Manturuk and I’m the Program Evaluator at Duke University’s Center for Instructional Technology. I get to evaluate a lot of interesting projects related to teaching and technology, but one of my biggest jobs is to evaluate Duke’s Massive Open Online Classes (MOOCs).
MOOCs are free, non-credit classes offered by universities and other institutions to anyone in the world. Since 2012, Duke has developed over 30 classes that have amassed over 2 million registrations. It’s my job to evaluate how well students in these classes are learning what the instructors want them to learn.
To accomplish this, I send out a lot of surveys – over 2 million surveys (and counting) in less than 3 years! When I started this project, I would be lucky to get 5% of people who registered for a class to fill out the pre-course survey, and the post-course survey response rates often hovered around 1%. It’s practically impossible to say anything evaluative with a 1% response rate, so I tried a lot of different things to get more people to fill out surveys. Some worked and some didn’t, but I learned several good lessons along the way.
Lesson Learned: When sending a survey to an online class, sign the email invitation with the class instructor’s name (with permission, of course). People are more likely to respond when the invitation comes from someone they know and respect.
Lesson Learned: Avoid sending surveys on Mondays when they are more likely to be ignored or accidently deleted during the first of the week email cleaning. It is better to send surveys from mid-morning on Tuesday through Friday.
Cool Trick: Tell people in the survey invitation how many questions they will be asked and how long it will take. I set it up so that students are asked just ten questions and then they are automatically brought back to their class.
Hot Tip: Be thoughtful about what demographic questions you ask in online classes. In some cultures, questions about race or gender are considered confusing, intrusive, or even offensive.
Rad Resource: Would you like to take a free class from Duke University? Visit our list of classes at https://www.coursera.org/duke. And if you do register for a class, be sure to fill out the course surveys!
The American Evaluation Association is celebrating DEOET TIG Week with our colleagues in the Distance Education and Other Educational Technologies Topical Interest Group. The contributions all this week to aea365 come from our DEOET TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to firstname.lastname@example.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.