STEM Program Evaluation Lab Week: Implementing EDI in the Current Evaluation Climate by Tyler Clark, Joy Alcantara, Omodolapo Somo-Aina, and J.R. Moller

Hello! We are Tyler Clark, Joy Alcantara, Omodolapo Somo-Aina, and J.R. Moller, graduate students in the UNC Greensboro Educational Research Methodology department program evaluation track. We participate in the STEM Program Evaluation Lab (SPEL) under the direction of Drs. Ayesha Boyce, Aileen Reid, and Tiffany Tovey. In this lab, we: 1) outline our community commitments/values for important discussions about social justice, 2) evaluate the impact of projects to broaden participation of underrepresented groups in science, technology, engineering, and math (STEM), and 3) participate in equity, diversity, and inclusion (EDI) training to develop and practice our professional skills.

Community Commitments: Although EDI is a foundational component of SPEL, the deaths of George Floyd and Breonna Taylor prompted the field of evaluation and further heightened our resolve to have discussions about social justice. We spent the last year discussing such travesties and the stressors that come along with them. We began by taking time to acknowledge our commitments, norms, values and expectations during these conversations. A few commonalities among responses were respecting others, confidentiality, reflection, participation, active listening, and open-mindedness. Having clear expectations allowed lab members to comfortably participate in these tough discussions. Ours are:

Professional Practice: We are inspired by the desire to evaluate efforts to improve the quality of STEM education and broaden participation in STEM fields, especially among groups that are systematically marginalized in STEM careers. Over the years, our lab has been able to support different STEM programs that are primarily focused on broadening participation in STEM fields through the Values Engaged, Educative (VEE) evaluation approach. We utilized VEE, a type of culturally responsive evaluation approach that assesses program quality at the intersection of cutting-edge scientific content, strong instructional pedagogy, and attention to diversity and equity. The approach is responsive to the context of programs and employs the multiple values and perspectives of stakeholders while attending to issues of social justice.

Professional Development: We have engaged in formal and informal training to further our understanding of our own perceptions of EDI and how it manifests in our evaluation work. We all participated in Fair Play training to help us understand and recognize different types of bias. This workshop and video game allowed us to “live” a graduate student’s experiences dealing with bias by recognizing the type of bias being encountered and choosing the response you believe he should have to it. Participation helped us understand the forms of bias we encounter, learn positive ways to address it, and understand the impact it has on one another.

Our lab also began to have an informal book club where we discussed How to Be An Antiracist. During our discussions we challenged our assumptions and personal beliefs while also questioning and expanding concepts brought forth by the book. We related ideas and concepts to our work as evaluators and discussed our role in dismantling systems of oppression found in our profession.

Lessons Learned:

• Greater self-awareness – This is critical to engaging others in a sensitive manner personally and professionally.
• More intentionality in language – This helps have more respectful dialogue in general and in evaluations.
• Centering evaluation in EDI helps to improve educational programs focused on broadening participation and improving quality.

Hot Tips:

• Discuss expectations and create a safe space for having tough discussions about the socio-political climate.
• Develop your own community commitments.

Rad Resources:

• Boyce, A. S. (2017) Lessons learned using a values?engaged approach to attend to culture, diversity, and equity in a STEM program evaluation. Evaluation and Program Planning, 64, 33?43.
• Bettez, S. C. (2020). Creating Community Commitments in the Higher Education Classroom. About Campus, 25(1), 14-18.
• Fair Play Game
• Greene, J. C., DeStefano, L., Burgon, H., & Hall, J. (2006). An educative, values?engaged approach to evaluating STEM educational programs. New Directions for Evaluation, 2006(109), 53-71.
• Kendi, I. X. (2019). How to be an antiracist. One world.
• Reid, A. M. (2020). Applying an educative approach to engage stakeholder values in evaluations of STEM research and education programs. Evaluation Journal of Australasia, 20(2), 103?108. DOI:10.1177/1035719X20918497

The American Evaluation Association is hosting STEM Program Evaluation Lab Week. The contributions all this week to aea365 come from evaluators working in this lab. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.