My name is Aaron Kates, and I am a PhD candidate in the Interdisciplinary PhD in Evaluation (IDPE) program at Western Michigan University. I am a psychotherapist, researcher, and independent evaluator.
“Has anyone ever had sex with you without your permission?”
The Guatemalan boy across the table from me was silent.
“Has anyone ever asked to take pictures of you in your underwear in exchange for money or anything of value?”
Quiet.
“I’m sorry. These questions are required by the government.”
He looked up at me. “Do you know how long I’m going to be here?”
It was my first job out of social work school. I was a therapist working with children who were fleeing Central America because of gang violence, abuse, or neglect. Most had accumulated a long list of traumatic experiences before making their way into our care. Looking back, I’m sure that the questions I asked upon meeting them the first time added yet another to that list.
Evaluation can be traumatic. Since moving on from this position, I don’t know how many refugee clients have told me, “I don’t want to talk about my past. I was asked about it too many times.” My work with these kids is just one example of how evaluation can be traumatic if not conducted with care for the participants. Below are some tips for avoiding these kinds of errors.
Tips for Avoiding Retraumatizing Pitfalls
#1: An Ounce of Prevention…
The simplest means for avoiding retraumatizing evaluation participants is to include them in evaluation design and implementation from the earliest stages. Adherents of Participatory, Utilization-Focused, Empowerment, and Social Justice Evaluation approaches incorporate this process. Including those who may be able to intuit the reactions of participants will safeguard the process from discovering later on that a particular question is offensive or retraumatizing.
#2: Engage in Developmental Meta-Evaluation
Developmental evaluation is an iterative process of evaluation that seeks to foster constant improvement, rather than seeking a summative judgment of a program. As evaluators, it is important that we adopt a similar stance to our evaluation work. We must never think that we have “arrived” at a perfect design. If a stakeholder tells us that an approach is traumatizing, we should be ready to listen and quick to change tack. Yes, it will be difficult. No, it won’t kill you.
#3: Follow a “Leave No Trace” Approach
Every good Boy or Girl Scout knows, when in the wilderness, “leave no trace.” Although an aspirational principle rather than an achievable goal, leaving nature as one finds it is of utmost importance in conservation. When working with traumatized populations, we should adopt a similar stance. Of course, we will have some effect whenever we collect data. But we should be collecting data with only a specific purpose in mind. Fewer data points mean fewer points of potential retraumatization. Before adding that question, ask, “Is this critical for the evaluation overall?”
Rad Resource:
Trauma-Informed Evaluation: Tip Sheet for Collecting Information
This week, AEA365 is hosting Trauma-informed Eval Week during which blog authors share lessons from and principles of trauma-informed evaluation. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hello Mr. Kates,
First off, I would like to start by saying that your article was an interesting read. It contained a lot of useful points and I found it easy to connect it to my own program evaluation and practice. From an evaluation standpoint, I never thought of it in a way like you stated here, “traumatic”…I couldn’t agree with you more that evaluation can be traumatic at times and that it is super important that we (as the evaluator) take this into consideration in any situation. Whether it be program evaluating like I am currently doing myself or evaluating students in the classroom.
“We must never think that we have “arrived” at a perfect design. If a stakeholder tells us that an approach is traumatizing, we should be ready to listen and quick to change tack” was a quote that stood out the most to me. In addition to what you have a stated, an outstanding evaluator must have strong analytical & intuitive skills and as you mentioned, you need to be able to “change on the fly”. This quote really made think about what I can do in my classroom to better my lessons or to just be a more effective educator. Just like you mentioned, if the stakeholders feels “traumatize”, you must listen and quickly make a change. This is so true in the classroom as well because if my students are not grasping information that I am teaching them then I must be “quick on my feet” and change my approach. It can be very traumatizing for students who are not understanding or grasping information. They can easily fall behind in their studies and therefore they begin to become more frustrated than what they were from the start.
With that being said, you have also brought another important point where you state that “Developmental evaluation is an iterative process of evaluation that seeks to foster constant improvement, rather than seeking a summative judgment of a program”. Again, connecting this to my career, it is important when assessing and evaluating our students that we focus on improving student abilities in their studies (e.g. teaching students to become lifelong learners) rather than focusing on just pure judgment of their work. As teachers, we assess and evaluate our students every day. I believe that when students understand that we are here to help improve their abilities and skills rather than judge will ultimately lead to much more success inside and out of the classroom.
Thank you for your article, it will help shaped my evaluation skills in my practice in the very near future.
Good afternoon Aaron,
My name is Lindsay Spence, and I am a high school teacher from Brooks, Alberta. In my school we have many students who are living with trauma. We have a large Immigrant population and include within that, many refugees. It is hard to fathom the trauma they have experienced in life and often, I worry that the content I teach within my social studies course is re-traumatizing for them. Before reading your article, I did not really think about the evaluation process being a traumatizing event itself. Knowing the backgrounds of the students in my room has always been beneficial, but it is nearly impossible to always do the right thing and watch for triggers in all the students all the time.
Your article was incredibly interesting from an evaluative approach. Not only does the utilization approach connect with real-time changes and accommodation to the students within the room, but the participator and empowerment evaluation focuses on self-determination and efficacy within the students from a self-evaluative approach. When it comes to students with Trauma, confidence and efficacy are a major issue and when they feel as though they are not successful, it triggers a response of flight or fight, rather than evaluation and reflection. To allow for confidence in “an ounce of prevention” with trauma informed evaluation, we may be able to alter how evaluation is approached by the students who have sustained trauma. The clients of the evaluation are center focus.
The Meta-evaluation approach would also lend towards self-reflection and resilience. If the students are more focused on their improvement rather than feeling judgement. Moreover, if the stakeholder tells us, as their evaluators, that the process is traumatizing, we should be willing to also take place in that improvement. That would not only model growth for our students but will better our own evaluative practices as well.
Finally, the “Leave no Trace” approach lends itself to purposeful data. Good evaluation, according to the Program evaluation standards, hold evaluators to propriety standards in which “evaluation will be conducted legally, ethically, and with due regard for the welfare of those involved in the evaluation, as well as those affected by its results” (pg. 1). If evaluation we are conducting is re-traumatizing our students, we are not ethically conducting the right type of evaluations. Moreover, if our evaluation is more focused, the students will be less overwhelmed and more confident in the completion of the tasks set out before them.
Thank you for your resource, it is targeted and useful and I will make sure I use this in my classroom.
Thank you for your thought-provoking article.
Lindsay Spence
Reference
The Joint Committee on Standards for Educational Evaluation, James R. Sanders, Chair (ed.): The Program evaluation Standards, 2nd edition. Sage Publication, Thousand Oaks, USA, p.23-24; 63; 81-82,125-126 (see http://www.wmich.edu/evalctr/jc/) or http://www.oecd.org/dev/pgd/38406354.pdf