Hello! My name is Amelia Ruerup, I am Tlingit, originally from Hoonah, Alaska although I currently reside in Fairbanks, Alaska. I have been working part-time in evaluation for over a year at Evaluation Research Associates and have spent approximately five years developing my understanding of Indigenous Evaluation through the mentorship and guidance of Sandy Kerr, Maori from New Zealand. I consider myself a developing evaluator and continue to develop my understanding of what Indigenous Evaluation means in an Alaska Native context.
I have come to appreciate that Alaska Natives are historic and contemporary social innovators who have always evaluated to determine the best ways of not only living, but thriving in some of the most dynamic and at times, harshest conditions in the world. We have honed skills and skillfully crafted strict protocols while cultivating rich, guiding values. The quality of our programs, projects, businesses and organizations is shaped by our traditions, wisdom, knowledge and values. It is with this lens that Indigenous Evaluation makes sense for an Alaska Native context as a way to establish the value, worth and merit of our work where Alaska Native values and knowledge both frame and guide the evaluation process.
Amidst the great diversity within Alaska Native cultures we share certain collective traditions and values. As Alaska Native peoples, we share a historical richness in the use of oral narratives. Integral information, necessary for thriving societies and passing on cultural intelligence, have long been passed on to the next generation through the use of storytelling. It is also one commonality that connects us to the heart of Indigenous Evaluation. In the Indigenous Evaluation Framework book, the authors explain that, “Telling the program’s story is the primary function of Indigenous evaluation…Evaluation, as story telling, becomes a way of understanding the content of our program as well as the methodology to learn from our story.” To tell a story is an honor. In modern Alaska Native gatherings, we still practice the tradition of certain people being allowed to speak or tell stories. This begs the question: Who do you want to tell your story and do they understand the values that are the foundation and framework for your program?
Hot Tip: Context before methods. It is essential to understand the Alaska Native values and traditions that are the core of Alaska Native serving programs, institutions and organizations. Indigenous Evaluation is an excellent approach to telling our stories.
Rad Resource: The Alaskool website hosts a wealth of information on Alaska Native cultures and values. This link will take you to a map of “Indigenous Peoples and Languages of Alaska”
The American Evaluation Association is celebrating Alaska Evaluation Network Week. The contributions all this week to aea365 come from AKEN members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Amelia Ruerup,
Good afternoon. I would like to take a moment to introduce myself. My name is Andrea Harding and I am a student enrolled in the Professional Masters of Education, with my field of study being Aboriginal Education, at Queen’s University in Kingston, Ontario, Canada. This semester I am taking a class called “Professional Inquiry and Evaluation”, and we have been asked to respond to a blog of our choice that can be found on the AEA 365 website.
I would first off, like to thank you for sharing your story. I am very interested to learn more about the similarities and differences in evaluation techniques used between the Indigenous Maori people vs. what it means to provide a culturally responsive evaluation in an Alaskan Native context.
Working in a Northern Canadian Indigenous community, I can appreciate the “Hot Tip” you provided that it is essential to understand the local Native traditions before providing evaluation.
In what context do you provide program evaluation? Has your program been adapted to use anywhere else in the world or with other Indigenous peoples?
The tradition of storytelling is a strong cultural resource. When using storytelling with the Alaskan people, what other traditional protocols do you adhere to? I am curious to know how it differs from the Yellowknife’s Dene First Nations people.
Thank you for sharing your Rad Resource which shows a map of Indigenous languages across Alaska. Here is a resource of the Northwest Territories Indigenous languages where I live.
http://www.pwnhc.ca/official-languages-of-the-northwest-territories/
In closing, I would like to thank you for your time,
Andrea Harding
Hello Amelia,
I was drawn to this post because I am currently developing an evaluation process for a program that provides apprenticeship training to Indigenous students. This is a pilot project in which we have reversed the (western) traditional training format. We are beginning our training in the Indigenous community of our partner and incorporating traditional teachings with innovative technology such as virtual reality welding simulators. This model allows us to complete the first module of training for the students in their own community so that when they attend their second module on campus, they will attend as a cohort and already have attained some of the technical skills.
As I read your post, I was struck with the importance of the storytelling aspect to evaluations. I had previous understanding of the commonality of oral storytelling to many Indigenous cultures, but I had not connected that understanding to the evaluation process in my own program. The stakeholders in our program include my post-secondary institution, our government funder, our corporate partner and our Indigenous partner. Given the various partners, an important consideration is who tells the story of this pilot project, because, as you state, “To tell a story is an honor” and there is still a tradition of only “certain people being allowed to speak or tell stories.”
I am wondering if you have any advice on how to best ensure that the story of our program is being told by the appropriate person (or people) and how to best ensure that the story is then shared in a manner and format that demonstrates cultural sensitivity. Does the sharing of the story become a collaborative effort by all of the stakeholders?
We have also been exploring the use of PhotoVoice as one method of allowing the students in our program to share their perspective of their experience. I understand that oral storytelling pre-dates photography by thousands of years, but I wonder what your perspective is about this visual method of storytelling, and its potential for program evaluation.
Thank you for your intriguing post!
Hello Amelia,
My name is Monika Kapica and I am currently a student of the Professional Masters of Education through Queens University. My program focus is on Aboriginal Education. This semester I am enrolled in four courses, two of which are Aboriginal Leadership and Knowledge in the School/Classroom, and Program Evaluation. Coming across your article was perfect timing, as I am currently working on a program evaluation design assignment (Aboriginal Education Strategy in Ontario) for one of my classes.
I was drawn to your article because the concept of Indigenous Evaluation is not very widely explored. I have found that there is a lack of Aboriginal people working in sectors of research and evaluation, and non-Aboriginal individuals often adopt Western approaches to evaluation. Prior to reading your response, I knew very little about what Indigenous Evaluation entailed, what was central to it, what commonalities different approaches had, and more importantly, how it differed from Western approaches to evaluation. I knew however, that storytelling is central to all Indigenous communities .I think that storytelling is a very important aspect of evaluation, because it encompasses meaning that cannot be derived from numerical data.
This course has been my first encounter with Program Evaluation, so everything is still very new to me. The concept of story telling reminds me of the development of a Program Theory in Evaluation. You mention that in order for proper evaluation to take place, a story must be told and known. This reminds me of the theory of action and change , as well as the logic model. These models suggest that in order to
understand how a program is going to achieve desired outcomes, it is essential to learn about the individuals which it is going to affect. Likewise with story telling, one can learn about the values that a program needs be rooted in, in order for it to be effective and meaningful to the community it is directed at.
This also reminds me of the BetterEvaluation Rainbow Framework. This framework has seven essential steps to evaluation: Manage, Define, Frame, describe, understand causes, synthesize, report and support use. Storytelling seems to me as similar to managing and defining and evaluation, because it entails learning the background information required to proceed.
On the AINHEC website which you have provided, one of the resources that stood out the most to me was the pdf on, “Cultural Considerations for Gathering Information”. Since evaluation is a complex process, I was wondering how information may be gathered to complete the evaluation. I see from this link that there are many methods including questionnaires, interns, focus groups, tests and observations. In your community, which procedures are the most common? Also, who designs these protocols? For example, who conducts the interviews or designs the questions in the surveys? I am interested to learn more about how the remainder of the process is designed and who is involved.
After doing a little bit more research, I found this website: https://www.acf.hhs.gov/sites/default/files/cb/tribal_roadmap.pdf
This website goes through the evaluation of a child welfare program situated in Alaska, and I found a unique figure on page 1 which highlights a framework for evaluation called a Strategic Roadmap. It is interesting because it incorporates historical contexts with the ‘new narrative that is to be established’ while also focussing on relationships and skill building. Are there any similarities and differences to the evaluation framework that you are most familiar with in your community?
After reflecting on this, some questions which I have are:
– How do Indigenous Australian and Indigenous North American evaluation
approaches differ How are they similar?
– In your community, how common is story telling ? Do youth participate in story
telling? Are youth engaged in story telling?
– How do you establish when, where, how and to whom (and by whom) stories are
told?
– After story telling takes place, how do you go about evaluating different programs
within your community? Is storytelling a continuous element in your approach?
– How do you go about deciding what changes need to be made if your program is
not resulting in desired outcomes.
– Is story telling also used to finalize an evaluation?
– How do you go about educating about the value of evaluation in your community?
– What has been your most challenging experience with Indigenous Evaluation? –
– What is the most challenging aspect of conducting evaluations in Indigenous
Communities?
– How do you go about re-engaging youth or elders that are not willing to participate
in program evaluation.
I hope this comment find you well! I look forward to hearing about any insights you may have, and how your journey in program evaluation has progressed over the past three years. I am very interested in this concept, and therefore have lots of questions to ask. Because I do not currently work with Aborignial people, I always look forward to learning from the experiences of the people who do!
Sincerely,
Monika Kapica
The information given is very thought provoking. From living in an Alaskan village, I completely accept and agree with her experience and research. I look forward to learning more about and seeing the results of, this system of approach!