Hello! We are Rebecca Teasdale, Lindsay Maldonado, and Cecilia Garibay—three Chicago-based evaluators who practice in museums, aquariums/zoos, libraries, and other informal learning contexts. Given that Chicago is known for its world-class cultural institutions, it’s a great home base for our work. Today, we’re pleased to share three lessons we’ve learned about evaluation of informal learning experiences.
Lessons Learned:
Developing methods. Evaluation of informal learning is a young and growing area of practice. As the field has come of age, we’ve learned that many of the evaluation methods developed for structured, compulsory learning environments are a poor fit for the self-directed, free-choice contexts we examine—prompting evaluators to develop new methods. Beverly Serrell developed some of the earliest museum-specific methods, including a criteria framework for exhibition evaluation and methods for examining how people move through exhibitions. Recently, an issue of New Directions for Evaluation (NDE) reported on current work to develop new, creative methods that are appropriate for these complex environments.
Common constructs. Informal learning experiences that focus on science, technology, engineering, and mathematics (STEM) often seek to foster STEM interest, engagement, and identity. But these are challenging constructs for evaluators to define and measure. Recently, we’ve drawn on interviews with leading researchers conducted by a task force with the Center for the Advancement of Informal Science Education (CAISE) to help us consider how to best conceptualize and study these constructs in STEM-focused and other informal learning contexts.
Equity and inclusion. Historically, museums have been oriented toward visitors from privileged segments of society. A recent report from the American Alliance of Museums and a toolkit and report from another CAISE task force highlight steps museums can take to foster greater inclusion and equity. We’ve seen the key role that evaluation can play in these efforts by foregrounding the experiences of communities that have been marginalized and using evaluation data to help program staff examine their practices and assumptions (check out this recent NDE chapter for examples).
Rad Resources:
To connect with colleagues who evaluate informal learning, we love AEA’s (recently re-named) Arts, Museums, and Audiences TIG and Environmental Program Evaluation TIG.
We’re also active in the Visitor Studies Association, which publishes the Visitor Studies journal, hosts an annual conference, offers webinars, and more.
Informalscience.org is an essential resource from CAISE that includes a repository of evaluation reports; an instrument clearinghouse; the interviews, report, and toolkit discussed above; and other Rad Resources.
The American Evaluation Association is celebrating Chicagoland Evaluation Association (CEA) Affiliate Week. The contributions all this week to aea365 come from CEA members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hello Rebecca, Lindsay and Cecilia,
Having worked in a science museum for several years, I was interested in how you described the experience of evaluating learning in informal settings. In the course I am taking right now, we are learning about evaluation, but mainly in a more formal program situation, often relying on social program examples to build our foundational knowledge. I recognize how the approaches I’m learning about would be challenging to apply to an informal learning experience, as these are complex environments with participants diverse in age, ethnic background, literacy levels, general interest, and educational background.
I appreciated the efforts of organizations you noted that are working towards improving equity and inclusion for museum visitors – in my travel experience, I’ve found that many museums outside of North America are either free (for everyone, or for people 25 and under), or have significantly lower admission fees, giving greater access to these experiences. This is a great way to encourage people of all backgrounds and financial means to engage in the informal learning that can take place in museums.
Some evaluation researchers have advocated for evaluation findings to have greater attention to seeking improvements to better serve marginalized groups, including having clients have a voice in participatory evaluations. These clients, though possibly difficult to engage, would offer unique and distinct orientations and perspectives than those of program or evaluation staff. Client-inclusive approaches would support a broader scope of the study and assist in addressing inequalities and access, in evaluating informal learning experiences.
Dear Rebecca, Lindsay & Cecilia,
I really enjoyed your discussion regarding STEM. I would also agree that this is challenging to evaluators to measure. STEM provides students with an opportunity to explore deep learning opportunities and to make new discoveries. I think it is a great thing to implement in the classroom and creates a move inclusive learning environment for students. I introduced a lot of STEM projects in the grade 5/6 classroom I was completing my practicum in during teachers college. The students loved this type of hands on learning and developed new and improved mathematical skills in the process.
I think that museums are a great informal learning context that help students explore and develop new ideas. Museums foster student engagement and provide varied learning opportunities for kids. I think that after evaluating informal learning, museums would be found to be beneficial to academic achievement. You discuss the idea of equity and inclusion through the use of museums etc. Many museums are free and accessible to the public, which creates a sense of community and inclusion for all.
As a new teacher, I enjoy implementing informal learning into my teaching practices. It is a fun way to engage learners and help them to develop growth mindsets. Learning does not always need to be formal. Instead, students will often learn more when they are engaged with the curriculum and can make connections to the real world.