PE Standards Week: A Delicate Balancing Act: Encouraging Stakeholders’ Participation in an Emotionally Charged Context by Rosalie Hirch & Brandi Geisinger

Hello! We are Rosalie Hirch and Brandi Geisinger, from the Research Institute for Studies in Education at Iowa State University. Sometimes we work with emotional situations for students who feel vulnerable, especially if they’re from traditionally marginalized groups. As a result, when we conduct organizational climate studies, we often find unique challenges in getting students to discuss their experiences. One approach we’ve taken to these issues is within the framework of the Program Evaluation Standards. In this post, we’ll focus on the delicate balancing act of respecting an individual’s rights (Propriety) while getting information that could lead to systemic change (Utility and Accuracy).

For one project, we were asked to evaluate the climate of a program at another university. Most such studies are instigated by faculty or administrators, but because this was initiated by students, the dynamics changed. Students felt empowered and were eager to share their experiences, though this led to emotional interviews. The key to balancing rights and getting information was to acknowledge students’ experiences without passing judgement and ask if they wanted to continue. In this case, all of the students did.

However, in other cases we’ve run into, students are more reluctant to share their experiences. In one, we were working with a department that retained few non-traditional students (i.e., female, African American, Latinx, LGBTQ+) and wanted to see to what extent department climate was a problem. In one focus group, an international student described an incident in which he and some friends were pelted with eggs, but immediately tried to downplay the severity. When we commented on the magnitude of the situation, he responded “If I think about it, I just get angry.” His experiences were important, but respecting his rights meant that we chose to pursue another topic.

Another situation from that study involved interviews with female students. In focus groups, we frequently heard that female students probably had the most complaints; however, we found females reluctant to share any negative experiences they’d had, possibly from fear of being identified. Thus, we chose to take an indirect approach, asking them if they had heard of any problems. After some hesitation, one student shared some rather shocking experiences she’d heard of, and the students began to open up more, possibly because they were able to distance themselves. This approach allowed us to collect useful and accurate information while respecting the rights of our participants to share stories from which they were more removed.

Lesson Learned:

The Program Evaluation Standards are guidelines for how to conduct all aspects of evaluation; for us, it has been important to keep in mind that Propriety must be balanced with Utility and Accuracy. One of the strengths of qualitative evaluation methods is hearing actual experiences rather than seeing people as numbers. For that reason, it is just as important to find ways to respect individuals’ human rights as it is to gather information that can lead to change.

This week, we’re diving the Program Evaluation Standards. Articles will (re)introduce you to the Standards and the Joint Committee on Standards for Educational Evaluation (JCSEE), the organization responsible for developing, reviewing, and approving evaluation standards in North America. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.