AEA365 | A Tip-a-Day by and for Evaluators

TAG | Native American

I’m Corrie Whitmore, an internal evaluator working for Southcentral Foundation. SCF is an Alaska Native Owned and Operated healthcare organization serving approximately 60,000 Alaska Native and American Indian people living in Anchorage, Matanuska-Susitna Valley, and 60 rural villages in the Anchorage Service Unit. Our organization has had program evaluation in-house since 2009, so our small department focuses on helping people in operations understand why evaluation matters and how it fits into what they do every day.

Hot Tip: Build relationships! Sometimes the most efficient way to get things done is not the best way to move the project forward – making time to listen, ask questions, and puzzle out what an evaluation will offer people “in the trenches” is very important.

Hot Tip: Get out of the office!  Going to the programs we work with and watching operations unfold builds trust with our customers, teaches us about their processes and data collection, and shows them we care about what they do.

Hot Tip: Ask concrete questions! It can be difficult for people to puzzle out logic models or identify program objectives, if they don’t have a background in that area, but most practitioners can confidently answer questions like:

  1. What does success look like?
  2. How do you know if things are going well?
  3. How do you know if something needs to change
  4. If you had a magic wand, what one thing would you change?
  5. What helps you make decisions today?

Hot Tip: Get something on paper – then tear it up! We use Anne Lamott’s idea of first drafts  to encourage writing things down early in the process. It’s much easier for our clients to identify what sounds appropriate and what feels “off“ once they have a document in hand to edit. Going through multiple drafts offers customers a chance to grapple with the language used, cultural appropriateness, and feasibility of the evaluation plan at all stages of the project, increasing their ownership of the final product.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · · ·

Hi, my name is Susan Geier and I am a doctoral student at Purdue University studying research methods, measurement and evaluation. I employ a participatory evaluation approach with the GEMscholar project and have learned much from the Native American college students and the dedicated program staff.

Lessons Learned: I would like to share my three R’s for participatory evaluation:

1. Build Rapport: In addition to conducting formal interviews and assessments, I interacted informally with the students and mentors when time allowed, during meals and in between activities. I spent time learning about Native American history and culture from the project team and students.

2. Demonstrate Relevance: I discussed with the stakeholders and participants possible benefits of the evaluation process and their unique roles in the improvement and success of the program components. For example, when the students expressed interest in helping future GEMscholars, a peer-mentoring option was added to the program. Consequently, students began to see the evaluation process as a mechanism for sharing their experiences and suggestions instead of an outside critique of their lives and activities.

3. Maintain Responsiveness: I provided the stakeholders with information in a timely and accessible format. Often these were oral reports followed by brief documents outlining the changes discussed. We had conversations about those issues that could not be resolved in a timely matter and possible effects on the program. In turn, the project team made ongoing changes, adding components where needed and modifying those elements that were not serving the objectives of the program. Assessments were modified if needed and the process continued.

Hot Tip: Journaling is a useful technique to capture real time reactions to interventions. This is particularly important when working with groups who are being introduced to unfamiliar and/or uncomfortable experiences as part of an intervention. I worked closely with the researcher and program coordinator to develop pertinent guiding questions for the students’ and mentors’ daily reflection journals. This is also a good time to develop an analysis rubric if applicable. Journals can be hand written or online (I provide a link to an online journal using Qualtrics). The journal entries provide a project team with valuable insights about how the program elements are perceived by all involved.

If you want to learn more from Susan, check out the Poster Exhibition on the program for Evaluation 2010, November 10-13 in San Antonio.

· ·


To top