My name is Jill Denner from Education, Training, Research (ETR), a non-profit organization that does research, evaluation, program development, and professional development. We partner with schools and colleges to increase the interest and capacity of girls and Latino/a students to pursue computer science.
Computer Science Education in K-12 is a relatively new space. It is a young discipline that is trying to distinguish itself from other Science, Technology, Engineering, and Mathematics (STEM) fields. And rightfully so. The “T” is different in many ways: There is less diversity in “T” classes and programs. Most programs do not have clear goals or a logic model to describe how their activities will lead to identified goals. There are many different learning outcomes, but few validated measures, established theories, or clear stakeholders who can drive key decisions about evaluation design, sampling, and measurement.
Hot Tips: Evaluation can make real contributions to a field that is in its infancy, but we need to know the stakeholders or audience and what they want to know. In CS Education there are different views, but most want to know who is benefiting and why/not. For example:
- Funders require evaluation to document return on investments. These include the US National Science Foundation, private foundations like the Motorola Solutions Foundation, and tech companies like Google.
- Educators or program developers want evaluation to help them make improvements in impact, design new programs, and get more funding
- Policymakers want what programs or policies to invest in
- Researchers want to inform theory, test hypotheses, and fill gaps in knowledge
Cool Tricks: How can you do good evaluation without established theories, logic models, or measures? This issue is particularly relevant for a field that places a high priority on increasing diversity. The following issues are important to consider when evaluating CS Education:
- Culturally responsive evaluation can help evaluators avoid perpetuating unconscious bias about the type of person who belongs in computing fields
- Getting demographic information is important, but asking students about their gender or race/ethnicity before questions about computing might trigger stereotype threat and affect responses
- Studying only individual factors misses the relational and institutional factors that affect participation and program impact
Rad Resources:
- Mark Guzdial’s blog to learn about issues that are central to Computer Science Education including his article on challenges that face Computing Education.
- Kimberly Scott and her colleagues have developed a theory of culturally responsive computing
- Talking points on unconscious bias in the classroom from the National Center for Women in IT can help evaluators avoid triggering stereotype threat
- Google’s recent reports on computer science education provide landscape data on key issues.
The American Evaluation Association is celebrating STEM Education and Training TIG Week with our colleagues in the STEM Education and Training Topical Interest Group. The contributions all this week to aea365 come from our STEM TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.