First, I’d like to introduce myself. My name is Persephone Hooper Lewis. I am a citizen of the Yomba Band of Shoshone Indians. The land that we now call home was historically our Springtime place. It is high in the mountains, nestled in the pines, smells of Behoveh (Great Basin Sagebrush) year-round, and is nourished by the sacred waters of the Reese River and winter snows. Some of the foods that we gather include buckberries, pine nuts, elderberries, chokecherries, and root vegetables. My families include the Kawich and Tutuwa Bands. I am married into the Lewis family from the Vah Ki district of the Gila River Indian Community. Second, I am also blessed to be invited by Drs. Rita S. Fierro and Kate McKegg, who generously and kindly organized and edited this series of blogs focused on decolonizing evaluation to ensure power structures are interrogated, researcher positionality is explicit, and the needs of marginalized communities are centered.
I have been working in Native American youth programs and education for the past twenty years. In this time, U.S. federal and state agencies were most often the funders and within these funding contracts were evaluation requirements that required our after-school programs to demonstrate impacts based on colonial definitions of success. The definitions frequently required us to collect standardized tests scores, attendance data, and the grade point averages (GPA) of the students with whom we served. The expectation was that the program we developed and implemented would increase the test scores and GPA while decreasing the number of days students were absent from school. After three years with a tribal program and repeated reports that included data that failed to meet stated objectives, my conversations with our external evaluator began to include more questions than answers. I wanted to know why we were continuously reporting data that was telling a different story than the changes I was seeing within our students? Why was the data never information that we could use on the ground? Why were we rarely, if ever, a part of the evaluation process beyond gathering data? And why did the reports fail to tell a story that the community would want told?
Pueblo writer Simon Ortiz has always argued that storytelling reflects the belief system of Indigenous people. He writes, “ Oral tradition is inclusive; it is the actions, behaviors, relationships, practices throughout the whole social, economic, and spiritual life process of people. In this respect, the oral tradition is the consciousness of the people.” Because methodologies form the theoretical foundations and subsequent tools that are used to paint a picture of “truth” within the evaluation process, data gathering methods should have a storytelling function. We must push against tools that offer stories based in deficit perspectives that continue to pathologize us and funders need to see their role in perpetuating these disempowering stories. Evaluators must ensure that the story we are telling about our children, community members, and Elders includes the beauty of our existence like the ways we are resisting coloniality, regenerating our cultural practices, and healing from over five hundred years of domination.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hi Persephone,
My name is Melissa and I am writing to you on the traditional territories of the W?SÁNE? People on beautiful Vancouver Island. I am an elementary school teacher and I’m currently doing my Professional Master’s of Education specializing in Indigenous Education through Queen’s University. As part of my Master’s, I am taking a course on Program Inquiry and Evaluation and that’s how I happened upon your post. Although I am quite new to the evaluation world, I’ve taken a real interest in ways that evaluators can decolonize the evaluation process.
What you mentioned about the data collection methods being centred around “colonial definitions of success” (e.g. standardized testing and attendance records) really hit home for me. In other coursework I’ve been reading about how colonized data collection (specifically in the education sphere) can reflect skewed results and can fail to accurately diagnose and articulate problems within systems or groups of learners; for example, many Indigenous learners get mis-diagnosed as having language delays based on the results of standardized language assessments that use Standard Canadian English without failing to take into account the fact that many Indigenous learners actually use an Indigenous Dialect of English. The data in that example certainly results in a story being told from a deficit perspective and completely “pathologizes” Indigenous learners. I think you are so smart to question why the data that was being reported was telling a different story than the changes you observed within your students. It really speaks to the importance of varying the data types (balancing qualitative and quantitative) in a decolonizing evaluation approach to get a more complete picture that embraces Indigenous brilliance and resilience, rather than being rooted in deficit thinking and trauma.
I loved your questions about why wasn’t the data being collected actually useful to you, the primary intended users of the evaluation, as well as why you weren’t involved in the evaluation process beyond the data collection phase. It sounds like a collaborative approach in which the community (and especially primary-intended users) is more involved at all steps of the evaluation process would have been beneficial. Much like moving toward qualitative data with a storytelling approach, increasing community involvement would better reflect a decolonizing approach (based on my understanding of it). I think it comes back to what is the ultimate goal for the evaluation, and the fact that evaluations should carry out the desired purpose for the primary-intended users. If the evaluation isn’t useful to the primary-intended users, then its approach should be reconsidered and changed with input from the users and community.
Thank you for sharing your experiences as an Indigenous evaluator. I wonder if your journey of decolonizing evaluation has been met with resistance or enthusiasm by those who you work with? I think decolonizing evaluation is incredibly valuable and important work, especially as we move toward reconciliation. Thank you for all the work that you are doing to further this process along!
Take care,
Melissa
Hi Persephone,
My name is Curtis Kennedy and I am a Vice-Principal/Teacher in Vancouver, BC who is currently completing a Program Inquiry an Evaluation course through Queen’s University (Canada) as part of a Masters of Education degree (PME). First off, I’d like to thank you for sharing your personal background and brief synopsis of your local area/culture.
I find the topic of decolonization of Indigenous education to be fascinating and something I have seen the need for this first hand, teaching in various First Nations communities. I found your inquiry questions to be very interesting, essentially why the data you were examining did not represent what your witnessing in real-time and why “reports fail to tell a story that the community would want told.” I have seen this as well in the past with data either being stale or inaccurate, a disconnect to what the current situation is, as well as possible agendas, as put, “Why were we rarely, if ever, a part of the evaluation process beyond gathering data.”
I connected with your idea that, “storytelling reflects the belief system of Indigenous people…(and) the oral tradition is the consciousness of the people.” As you later elude to, this is significant aspect and first step to approaching the decolonization of Indigenous education. To go further, you stated, “because methodologies form the theoretical foundations and subsequent tools that are used to paint a picture of “truth” within the evaluation process, data gathering methods should have a storytelling function.” How would you utilize storytelling as a form of program evaluation? What are some other strategies you would suggest to further decolonize Indigenous education?
Thanks for sharing!
Curtis Kennedy
Hello Persephone,
I am an educator in Kamloops, British Columbia at a beautiful little school known as Four Directions Secondary. Four Directions Secondary School provides an alternate learning environment designed to serve secondary students of Aboriginal ancestry who are more likely to succeed in a smaller, more personal and flexible setting. In addition, the program includes a significant focus on Aboriginal culture and a variety of out of school activities. I am currently enrolled in a Program Inquiry and Evaluation course at Queens University as part of my Professional Master of Education. I have been working on an evaluation design plan for our school’s after-school program known as the Community Led Learning Program. I was so intrigued and inspired when I came across your article!
First of all, thank you for sharing your story and painting a beautiful picture of your home land. I can tell right away that you have a strong sense and respect of place. Your work in Native American youth programs and education sounds inspirational. I love that you had the courage and respect to ask such essential questions about the meaning and intent behind the program evaluations. I agree with you that it is unfair and not representative to evaluate programs based on colonial definitions of success. Thank you for helping to raise awareness about the importance of using multiple perspectives and techniques to incorporate diversity, equity and inclusion into the evaluation process. I hope that the significant questions that you so bravely asked help to pave a change in a colonialist system of evaluation.
I was wondering if the evaluators that you had experience with helped to incorporate your questions and advice into the evaluation plans? I surely hope this is the case, as evaluations should abide by the standards to ensure that programs are being evaluated fairly and equitably. I hope that your strength and determination help create a shift in evaluation planning, processes and reporting that better reflect Indigenous ways of knowing, learning and being.
Thank you for sharing this empowering article with us! I look forward to sharing it with my peers and colleagues.
Sincerely,
Courtney Bruin
Hi Persephone,
My name is Emma Wilson and I am a secondary school teacher in Vancouver BC who is currently enrolled in a course on evaluation at Queen’s University as part of my Professional Masters of Education. Thank you for taking the time to share your perspective with your readers! The way you describe your home was breathtaking, and I was deeply moved by your piece.
I have often lamented the way our current educational system is organized to favour standardized testing and GPA scores. It has always seemed to me that this isn’t an accurate way to properly demonstrate learning, and it leaves behind so many people who do not fit into this educational mold which has been pushed on students for centuries. How can we properly demonstrate learning and growth when all we have to measure is memorized facts that will be forgotten the moment the book closes? It has been a question that I have been grappling with in my time as both a teacher and a student, and I expect I will continue to do so throughout my career.
I have to admit that in my own ignorance I never attributed this system as being the product of colonialism, and reading you describe it as such caught me off guard. But it has all become so clear to me now that this is yet another way in which colonialism affects all aspects of life here in North America, and particularly in the lives of Indigenous people. It’s so unfair that the youth that you work with are being underrepresented because of these colonial systems. There is so much more to consider when evaluating a group to determine growth and the obsession that society places on test scores fails to consider the whole story. This is yet another way in which Indigenous people suffer at the hands of colonial practice. This generational trauma and subsequent journey towards healing is so rarely taken into account in these evaluations. You’re right in that the focus should be on how the communities are thriving and growing despite the immense pain and suffering that has occurred, and continues to occur because of colonial influence.
This piece of writing was incredibly powerful and it allowed me to examine the ways in which evaluation is performed and how we need to do better. It’s so unfair to place so much emphasis on these arbitrary measures of learning when there is so much more to consider. I hope that this will change and that evaluators can make strides to rectify this.
Thank you so much for sharing,
Emma
This is powerful. Thank you for enlightening and encouraging me (like Eval20 did) to continue to think outside the normal evaluation box for more inclusive and accurate practices.