Climate Ed Eval week; Rachel Becker-Klein on Using Embedded Assessment Tools for Evaluating Impact of Climate Change Education Programs on Youths

My name is Rachel Becker-Klein and I am an evaluator and a Community Psychologist with almost a decade of experience evaluating programs focused on STEM (science, technology, engineering, and math) education, citizen science, place-based education, and climate change education. I’ve worked with PEER Associates since 2005. PEER Associates is an evaluation firm that provides customized, utilization-focused program evaluation and educational research services for organizations nationwide.

Recently I have become very interested in thinking about how to measure impacts of climate change and other STEM programs on youths. These programs often use out-of-school time and outdoor settings to teach important content and skills, so using traditional surveys and standardized tests may not be appropriate ways of assessing youth learning. Embedded assessment offers an innovative way of capturing student content knowledge, skills, and science dispositions that can complement traditional standardized tests and surveys in formal educational settings. Embedded assessments are one form of alternative assessments and can be defined as “opportunities to assess participant progress and performance that are integrated into instructional materials and virtually indistinguishable from day-to-day program activities” (from page 184 of Wilson & Sloane). This assessment technique allows learners to demonstrate their STEM/climate change competence in informal settings without undermining the voluntary nature of learning in such settings.

Lessons Learned: While there is considerable interest in embedded assessment (as gauged by talking to other evaluators and reviewing assessment literature), there are few published articles that examine these assessment strategies for their validity, reliability, or correlation with more traditional assessment techniques. There seems to be a strong need for understanding more about how embedded assessment approaches can be used for climate change and STEM education.

Designing embedded assessment is challenging, takes a lot of time, and requires close collaboration with program staff in order to ensure that the task is truly embedded into the program activities. However, when done well, it seems worth the effort.

Cool Trick: One example of an embedded assessment is for a project that PEER evaluated which included teaching youths how to develop and create their own Augmented Reality (AR) games. In order to assess youth skills at AR game development, we created an AR challenge activity for them to do, with an accompanying assessment rubric that evaluators used to assess youth skill level.

Rad Resource:

The American Evaluation Association is celebrating Climate Education Evaluators week. The contributions all this week to aea365 come from members who work in a Tri-Agency Climate Education Evaluators group. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

3 thoughts on “Climate Ed Eval week; Rachel Becker-Klein on Using Embedded Assessment Tools for Evaluating Impact of Climate Change Education Programs on Youths”

  1. Emily Nishikawa

    Hi Rachel,

    I am a student at Queen’s University in the Professional Master of Education program, and I found your post very interesting as it relates closely to my passion for Outdoor Education. I have worked as an instructor in a Forest School, and am now a Grade 2 teacher in the public school system. I am incorporating outdoor learning into the curriculum but would love to bridge the gap to have more of a Forest School approach as my primary approach to teaching. A question I have is how best to demonstrate that this type of learning and instruction is as valuable as so many of us know it is so that stakeholders can support this type of learning within the school system.

    As an elementary school teacher who incorporates outdoor, place-based learning into my curriculum, I often find I am relying on conversations and observations as my primary forms of assessments to ensure students are meeting learning objectives while we are engaged in outdoor learning. We often come back from an outdoor experience and write about, or connect it to other learning in the classroom, to produce some sort of concrete evidence of learning. Your article about embedded assessment made me curious about the ways we can demonstrate learning such as becoming involved in community projects that demonstrate deep understanding and connection and a sense of stewardship. I often wonder if stakeholders will acknowledge other approaches to assessment as being as valid as our typically long-standing beliefs that tests and essays are the gold standard for assessment. In terms of program evaluation, having outcomes which are not as concrete might make evaluations more subjective, therefore potentially more unlikely to gain full support from all stakeholders. I hope that Outdoor Education, Climate Change and Environmental Education can work towards improving assessment opportunities to show just how important and impactful the learning is.

    Thanks for your post, and I would love to hear your thoughts and ideas about embedded assessment in Outdoor Education.

  2. Emily Nishikawa

    Hi Rachel,

    I am a student at Queen’s University in the Professional Master of Education program, and I found your post very interesting as it relates closely to my passion for Outdoor Education. I have worked as an instructor in a Forest School, and am now a Grade 2 teacher in the public school system. I am incorporating outdoor learning into the curriculum but would love to bridge the gap to have more of a Forest School approach as my primary approach to teaching. A question I have is how best to demonstrate that this type of learning and instruction is as valuable as so many of us know it is so that stakeholders can support this type of learning within the school system.

    As an elementary school teacher who incorporates outdoor, place-based learning into my curriculum, I often find I am relying on conversations and observations as my primary forms of assessments to ensure students are meeting learning objectives while we are engaged in outdoor learning. We often come back from an outdoor experience and write about, or connect it to other learning in the classroom, to produce some sort of concrete evidence of learning. Your article about embedded assessment made me curious about the ways we can demonstrate learning such as becoming involved in community projects that demonstrate deep understanding and connection and a sense of stewardship. I often wonder if stakeholders will acknowledge other approaches to assessment as being as valid as our typically long-standing beliefs that tests and essays are the gold standard for assessment. In terms of program evaluation, having outcomes which are not as concrete might make evaluations more subjective, therefore potentially more unlikely to gain full support from all stakeholders. I hope that Outdoor Education, Climate Change and Environmental Education can work towards improving assessment opportunities to show just how important and impactful the learning is.

    Thanks for your post, and I would love to hear your thoughts and ideas about embedded assessment in Outdoor Education.

Leave a Reply to Spokane Web Designer Cancel Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.