We are Oregon State University’s STEM Research Center (SRC), a team of dedicated professionals of various disciplinary backgrounds who conduct applied research and evaluation on STEM learning and education, science communication, and STEM engagement (figure 1). Working at the intersection of research, practice, and policy, we focus on equity and social justice and strive to understand learning across the lifespan. We use evidence-based strategies to investigate STEM engagement and education in formal and informal settings. Our partners utilize our work and empirical evidence to guide learner experiences and professional practice.
In this blog, we want to highlight one of our recent evaluation efforts with the NASA Heliophysics Education Activation Team (HEAT). HEAT comprises NASA scientists, engineers, and educators who aim to make heliophysics accessible to multiple audiences and learners of all ages. This year HEAT focused on education, engagement, and outreach for the total Eclipse on April 8th, 2024. In particular, NASA HEAT supported an event at the Dallas Arboretum in Dallas, Texas. Over 8,000 people experienced the eclipse at the Dallas Arboretum and engaged with NASA science and missions through interactive activities. SRC staff Heather Fischer, Martin Storksdieck, Victoria Sellers, Kim Preston, Ben Archibeque, and Matt Nyman supported evaluating HEAT’s eclipse engagement and outreach efforts.
Our evaluation assessed the value NASA adds to public engagement events like the 2024 Total Solar Eclipse programs. We focused on the degree to which NASA-related activities were engaging and educational and, specifically, the degree to which event attendees know eclipse safety precautions and basic eclipse mechanics. We used a mixed-methods approach that included spot interviews, surveys, and embedded assessment activities to address the evaluation questions with a typical sample of event visitors.
Summary of Key Findings
About eight in ten (79%) attendees we surveyed and interviewed said this eclipse is the most awe-inspiring event they have ever seen. From the interviews,select-all responses indicate that the public found NASA events informative (22%), interactive (19%), and thought-provoking (17%). People who participated in our mapping activity indicated that their favorite stations had more interactive activities, such as decorating eclipse glasses and corona chalk art. Attendees also enjoyed the stations with “artifacts/equipment” they could interact with, such as the sounding rocket nose cone.
One of NASA’s goals was to ensure the public understood how to view the eclipse safely. We found that nearly all (99%) of survey respondents correctly understood that they should wear special solar-viewing glasses, and (93%) of interactive activity participants understood that it is safe to remove your solar-viewing glasses during totality.
Lessons Learned
Our evaluation of this eclipse event has inspired the center to think more creatively about how we conduct evaluation, especially in settings like this one that require rapid data collection from large groups of people at large public events.
- In such settings, there is very little time to collect quantitative data through large surveys, detailed demographic data, or extensive interviews that can be used to judge the impact of events and/or evaluate participant satisfaction.
- There is limited time to interact with people to complete large surveys, and people generally have little patience for extensive interviews. We do not want to detract from participants’ enjoyment and engagement at the events.
- BUT, collecting useful and reliable data is still a priority to gauge participants’ satisfaction and the impact of these events, especially where clients have invested considerable time, financial, and material resources in developing and implementing an event.
We will be attending AEA in Portland, Oregon, we hope to see you there and answer any questions you may have about our work.
We’re looking forward to the fall and the Evaluation 2024 conference with our colleagues in the Local Arrangements Working Group (LAWG). Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to contribute to AEA365? Review the contribution guidelines and send your draft post to AEA365@eval.org. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.