Welcome to STEM TIG week! My name is Jason Ravitz and I conduct research and manage evaluations of educational outreach projects at Google. This week our blogs focus on Computer Science (CS) education which may officially be counted as a STEM field by act of Congress.
The CS First project is one curriculum that is available for use by schools, camps and after-school programs. In one case, Google has funded Boys and Girls Clubs of America to deploy AmeriCorps VISTAs to build capacity and use this curriculum in summer camps and afterschool clubs. The goals of CS First include increasing confidence and providing a sense of belonging in technology for underrepresented students.
This has been exciting work, but I’ve found there are challenges when evaluating informal STEM and CS programs. Not least is having to make teachers or volunteers test for pre-post content learning. This feels to everyone like it defeats the purpose, that is to have fun and not feel like school.
Cool Trick: Try to make assessment instructional and fun. We chose 5 basic level assessment items for a pre-test and asked volunteers to acquiesce to try these one time and report how it went. Meanwhile, to make these assessments less burdensome I came up with a way to punctuate each question with a fun activity to illustrate what kids would be learning. We had various ideas, like playing “Simon says…” as a way of demonstrating commands and loops. These would work even better with clickers and maybe as part of the curriculum. There are more vetted activities with accompanying research at CS unplugged. This is a “cool trick” because it can make assessment a learning activity that feels less like school. Even if our ideas weren’t generally used with the pre-tests, they showed we were listening to concerns about over-testing and its potential impact on what should be a fun club climate.
Hot Tip: Plan ahead with the curriculum provider. We are coordinating with the curriculum developer to incorporate information from their assessments and produce reports. This is non-trivial, but will be very beneficial for our evaluation. Among its assessments CS First has a way of scoring students’ code. The system they use, that you can try online, is Dr. Scratch, as developed by researchers in Spain. We hope our external pre-post tests can quickly be used to validate the embedded assessments (or refine them) so our external assessments can be retired.
We all want to hear what challenges others are facing and to hear your solutions. Thanks to Kimberle Kelly and AEA’s STEM Education and Training TIG for help organizing this week of blogs. Please join us at our conference sessions and tell us what you think!
The American Evaluation Association is celebrating STEM Education and Training TIG Week with our colleagues in the STEM Education and Training Topical Interest Group. The contributions all this week to aea365 come from our STEM TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.