AEA365 | A Tip-a-Day by and for Evaluators

TAG | indigenous learning

Hello! I’m Martha A. Brown President of  RJAE Consulting. Lately, an endless stream of conference speakers, blog writers, Indigenous evaluators, and authors have confronted and challenged my “programming” as an evaluator. Traditional evaluation methods place tremendous emphasis on research methods and evaluation theory – but not necessarily on the people we work with and for. At the 2017 Canadian Evaluation Society conference, Nora Roberts told me that the very tools of our profession continue to oppress and silence others. Her statement sent me reeling. Gail Barrington spoke about the value of reflecting upon our work and our methods so we can improve our craft and learn more about ourselves. Indigenous speakers at multiple conferences reminded me that we are all interconnected and that our relationships with ourselves and each other are the most important things in life. All of this can be summed up in one word: love.

Additionally, I research, practice and teach restorative justice, which is grounded in Indigenous values such as interconnectedness, openness, honesty, vulnerability, and respect. I bring these values and restorative practices to my work. However, too many times I have felt like I am “breaking all the rules” that I learned in graduate school as I infuse love into my work and the people I work with.

When I read the invitation to submit a blog on evaluation and labor, the first thing that came to mind was to write about putting love and relationships at the center of our work. What would our work look like if each of us took time at the outset and throughout every evaluation to build trusting relationships with our “stakeholders” and “participants”? Do those of us who are products of Western culture even know how to do this? In a society that values goals, outcomes, and return-on-investment above all else, how can we return to the teachings and the ways of our ancestors and put our relationships at the center of everything we do? We knew this once, but have forgotten.

In AEA, many evaluators are truly committed to changing the world, to improving people’s lives, and to creating more just and equitable ways of doing what we do. But we don’t always know how to live out our goals. That requires us to critically reflect upon what we were taught, how we do our work, and to ask who is being inadvertently silenced, harmed, or oppressed during an evaluation – or in an evaluation classroom. It requires us to love.

Love requires us to engage our whole selves – mind, body, heart and spirit – in our work. We can learn how to do this by studying Indigenous values, practices, and ways of being. I am so grateful to those who helped me wake up, including our own Nicky Bowman.

Rad Resources:

The American Evaluation Association is celebrating Labor Day Week in Evaluation: Honoring the WORK of evaluation. The contributions this week are tributes to the behind the scenes and often underappreciated work evaluators do. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Hello! My name is Amelia Ruerup, I am Tlingit, originally from Hoonah, Alaska although I currently reside in Fairbanks, Alaska.  I have been working part-time in evaluation for over a year at Evaluation Research Associates and have spent approximately five years developing my understanding of Indigenous Evaluation through the mentorship and guidance of Sandy Kerr, Maori from New Zealand.  I consider myself a developing evaluator and continue to develop my understanding of what Indigenous Evaluation means in an Alaska Native context.

I have come to appreciate that Alaska Natives are historic and contemporary social innovators who have always evaluated to determine the best ways of not only living, but thriving in some of the most dynamic and at times, harshest conditions in the world.  We have honed skills and skillfully crafted strict protocols while cultivating rich, guiding values.  The quality of our programs, projects, businesses and organizations is shaped by our traditions, wisdom, knowledge and values.  It is with this lens that Indigenous Evaluation makes sense for an Alaska Native context as a way to establish the value, worth and merit of our work where Alaska Native values and knowledge both frame and guide the evaluation process.

Amidst the great diversity within Alaska Native cultures we share certain collective traditions and values.  As Alaska Native peoples, we share a historical richness in the use of oral narratives.  Integral information, necessary for thriving societies and passing on cultural intelligence, have long been passed on to the next generation through the use of storytelling. It is also one commonality that connects us to the heart of Indigenous Evaluation.  In the Indigenous Evaluation Framework book, the authors explain that, “Telling the program’s story is the primary function of Indigenous evaluation…Evaluation, as story telling, becomes a way of understanding the content of our program as well as the methodology to learn from our story.” To tell a story is an honor.  In modern Alaska Native gatherings, we still practice the tradition of certain people being allowed to speak or tell stories.  This begs the question: Who do you want to tell your story and do they understand the values that are the foundation and framework for your program?  

Hot Tip: Context before methods.  It is essential to understand the Alaska Native values and traditions that are the core of Alaska Native serving programs, institutions and organizations.  Indigenous Evaluation is an excellent approach to telling our stories.

Rad Resource: The Alaskool website hosts a wealth of information on Alaska Native cultures and values.  This link will take you to a map of “Indigenous Peoples and Languages of Alaska”

The American Evaluation Association is celebrating Alaska Evaluation Network Week. The contributions all this week to aea365 come from AKEN members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Hello, we are Katherine Tibbetts and Wendy Kekahio, and both program evaluators doing work within the field of education in Hawai`i. Our work involves using indigenous ways of teaching and learning to inform culturally relevant and responsive ways of conducting research and evaluation studies.  One of our recent projects involved working with Hawaiian-focused charter schools to assess the impact of participation in professional development programs.

The Collaborative Inquiry (CI) project was designed to be culturally relevant and responsive–representing the values of relevance, rigor, respectful relationships, and reciprocity (for more information see, among others, Tibbetts, Faircloth, Villegas and Wheeler (2008)), The CI project extended the conventional purposes of evaluation to prove or improve, by employing a meta-action-research strategy to support the transfer of knowledge and skills learned at the training and assess their impact on teaching and learning. To do this, all participating teachers were required to conduct collaborative inquiry projects. They were encouraged to do their projects in small groups. The charter school teachers’ projects were supported by faculty contracted from a local college of education and culminated in a Ho`ike (demonstration of knowledge or skills).

Hot Tip: Supporting the Inquiry Projects. If you are interested in replicating this approach, it is important to provide ongoing support and scaffolding for the inquiry projects. The simplified action research curriculum and tools provided by the college of education faculty brought what were previously largely abstract concepts to life for the charter school teachers. Multiple “touch points” throughout school year, including visits to the charter schools enabled the college of education faculty to provide advice on the feasibility of project plans to identify potential sources of data that were tailored to each action research project, and helped sustain the momentum of the projects

Hot Tip: Assessing the Impact. As evaluators, our primary challenge was to synthesize information across a wide variety of projects. In the first year, there were 8 projects conducted in 3 different schools with topics spanning nutrition education, behavior management, mathematics, and writing. We approached the analysis as a multiple case study (based loosely on Stake, 2008) and ultimately created a rubric based on the CI project objectives and standards of inquiry. This allowed us to assess and summarize the quality of the inquiry projects.

Rad Resources:

Deloria Jr., V., & Wildcat, D.R. (2001). Power and place: Indian education in America. Golden, CO: Fulcrum Resources.

Hood, S., Hopson, R. K., & Frierson, H. T. (2005). The role of culture and cultural context: a mandate for inclusion, the discovery of truth and understanding in evaluative theory and practice. Greenwich, CT: IAP

Smith, L. T. (1999). Decolonizing methodologies: research and indigenous peoples. London: Zed Books.

Stake, R.E. (2006). Multiple case study analysis. New York: Guilford Press.

Thompson-Robinson, M., Hopson, R., & SenGupta, S. (Eds.). (2004). In Search of Cultural Competence in Evaluation (Vol. 102). Fairhaven, MA: Wiley Periodicals.

Tibbetts, K. A., Faircloth, S., Villegas, M., & Wheeler, L. (2008). Section III: Indigenizing accountability and assessment. In M. K. P. A. Benham (Ed.), Indigenous Educational Models for Contemporary Practice:  In Our Mothers Voice II. New York: Routledge.

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

· ·

Archives

To top