We are Courtney Blackwell, Heather King, and Jeanne Century from Outlier Research & Evaluation at the University of Chicago. For the last 3.5 years, we have been researching and evaluating computer science education efforts.
Computer Science (CS) is becoming a buzzword in education, with educators, policymakers, and industry developers promoting CS as key to developing 21st Century skills and a pathway to employment. While CS is not new to education, the spotlight on it is. In 2014, over 50 U.S. school districts, including the seven largest, pledged to make CS education available to all students.
Like all buzzwords, most people have their own vague idea of what CS means, but even experts working within CS education do not, yet, have a clear, agreed-upon definition. If evaluators are going to be able to accurately measure the effects of CS education efforts on teaching and learning, and accumulate knowledge and understanding, we need to have a clear definition of what “CS education” is. Until CS educators create shared definitions themselves, we, as evaluators, can do our part by ensuring our logic models, strategies, and measures clearly and specifically describe the innovation — computer science education — so that our work can inform others and further the field.
Lessons Learned: Evaluating an ill-defined intervention is not an uncommon problem. In the case of CS, however, the capacity to articulate that definition is limited by the state of the field. As evaluators, we have to find alternatives. In our evaluation of the Code.org’s computer science education efforts, we ask students to provide their own definition of CS at the beginning of our questionnaires. Then, we provide a specific definition for them to use for the remainder of the questionnaire. This way, we capture student interpretations of CS and maintain the ability to confidently compare CS attitudes and experiences across students. Similarly, we begin interviews with teachers, school leaders, and district leaders by asking, “How do you define computer science education?”
Hot Tip: Always ask participants to define what they mean by computer science.
Rad Resources #1: A recent survey by the Computer Science Teacher’s Association (CSTA) found that high school leaders don’t share a common definition of CS education. This suggests that school leaders may promote their schools as providing “computer science” when in fact they are providing activities that would fail to be considered CS at the college and professional levels.
Rad Resources #2: Check out LeadCS.org, a new website about to be launched, for definitions of key terms in Computer Science education. The website offers a range of tools for K-12 school and district leaders and their partners who seek to begin or improve CS education programs.
The American Evaluation Association is celebrating STEM Education and Training TIG Week with our colleagues in the STEM Education and Training Topical Interest Group. The contributions all this week to aea365 come from our STEM TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.