STEM TIG Week: Wendy DuBow on Sharing Survey Instruments: We’ll Go First!

Hi, I’m Wendy DuBow, a senior research scientist and director of evaluation at the National Center for Women & Information Technology (NCWIT). Our mission is to increase the meaningful participation of women in technology fields. We focus on sharing theory- and evidence-based practices with stakeholders in education and industry to support them as they recruit, retain, and promote girls and women in tech. In my position, I see a lot of K-20 interventions aimed at increasing women in tech, and alongside, a wide variety of measurement instruments.

Lesson Learned: Using Social Cognitive Career Theory. Most of the evaluations I see don’t take advantage of theory or past empirical evidence to ground their assessments. It would be great to share more theory- or evidence-based evaluation approaches. The social cognitive career theory (SCCT) model has been widely used to explain people’s educational and career interests in STEM. We wanted to specifically assess students in computer science-related programs, so we developed an instrument that uses SCCT to assess five constructs: interest, self-efficacy, outcome expectations, perceived social supports and barriers, and intent to persist in computing. Our survey has been used in a number of different educational settings, with middle and high school students, and with college and above. Of course, there are many other valid and reliable instruments available to evaluators of STEM education programs, but it can be hard to find them when you’re pressed for time in the proposal writing or instrument development stages. For expediency and for the larger good of sharing data and measuring interventions systematically, I would very much like to see STEM education evaluators and researchers have a shared repository of instruments. To this end, I’m holding two sessions at the Chicago AEA meeting to discuss

Hot Tip: Our SCCT survey instrument is publicly available upon request.

Cool Trick: We currently use SurveyMonkey for online surveys, and also have access to Qualtrix, so if you use either of these tools, we can share our SCCT survey directly with your pro account, already formatted though you can customize as you see fit! We just ask that you acknowledge NCWIT in any presentations or write-ups of the data.

Rad Resource: A variety of STEM assessment tools are already collected in the Engineering field:

Lessons Learned: Be sure that all of the SCCT survey constructs match the intended outcomes of the program, and tailor the wording of the parenthetical explanations of each item to the program being evaluated.

Get involved: Please come to the AEA 2015 Think Tank “Improving the Quality and Effectiveness of Computer Science Education Evaluation Through Sharing Survey Instruments” and the multi-paper session “Four Approaches to Measuring STEM Education Innovations: Moving Toward Standardization and Large Data Sets.”

The American Evaluation Association is celebrating STEM Education and Training TIG Week with our colleagues in the STEM Education and Training Topical Interest Group. The contributions all this week to aea365 come from our STEM TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.