Hello! My name is Julia Li, and I am a senior researcher at Rockman et al Cooperative, an education research and evaluation firm that specializes in STEM education. In recent years, I’ve seen a growing interest in STEM programs incorporating art-based lessons as a means to inspire more STEM engagement, learning, and identity in youth and adult audiences. The inclusion of art in STEM programs can create new evaluation challenges as art is an expansive genre and subjective to interpretation.
The evaluation of STEM + Art (STEAM) programs can be intimidating at first because there aren’t many standardized resources (e.g., surveys, assessments) for evaluating art, and the type of art used in each program can vary greatly. Recent findings from a couple of our ongoing STEAM projects have also revealed that the type of impact we’ve seen from STEAM projects may not fulfill the same expectations as more traditional STEM programs. For instance, including art-based lessons may not change students’ standardized test scores, but it may boost students’ confidence and science identity to increase their motivation to learn more about STEM topics. As the STEAM programs gain more ground in our field, it’s important for our evaluation plans to adapt to the changing needs. Hopefully some of our Lessons Learned and Hot Tips can help you prepare for your next STEAM program evaluation!
Lessons Learned
Define alternative measures of impact or success. Understanding the impact of STEAM programs may require shifting your evaluation lens to examine different outcome measures. The art component of a program may not directly effect changes in STEM content knowledge or skills, but the STEAM program could produce significant changes in STEM interest, engagement, problem-solving, and confidence measures. Greater interest or confidence can lead to improved STEM identity and motivation to learn. It’s important to think about other ways of defining success in STEM.
Examine impact on other audiences. Many school-based programs focus on how the curriculum supports learning and achievement in youth audiences, but STEAM programs can also impact educators and facilitators. At the beginning of the programs, educators often report that they lack the skills to lead art-based lessons, but their experiences in the program gave them more confidence in their ability to utilize new tools and strategies. Using art-based lessons helped educators find new ways of reaching students who struggle to express themselves or participate in class, and gave educators the confidence to try different teaching approaches.
Hot Tips
Include external expertise. Evaluating STEM programs with an art component can be challenging for evaluators who don’t specialize in different art mediums. Evaluators should consider including external experts – such as local artists – to be an evaluation partner. They can provide guidance on how to define, incorporate, and interpret different forms of art.
Expand the program audience. Additionally, art can be a means to communicate with a broader audience. Evaluators could expand the reach of the project by inviting participants’ families and community members to participate and reflect on the STEAM projects. Showcasing their STEAM projects to community members can help youth participants demonstrate their knowledge, skills, and confidence they’ve gained through the program. In addition, the presentations might spark greater community interest and involvement in STEM projects.
The American Evaluation Association is hosting STEM Education and Training TIG Week with our colleagues in the STEM Education and Training Topical Interest Group. The contributions all this week to AEA365 come from our STEM TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.