Hi, my name is Susan Geier and I am a doctoral student at Purdue University studying research methods, measurement and evaluation. I employ a participatory evaluation approach with the GEMscholar project and have learned much from the Native American college students and the dedicated program staff.
Lessons Learned: I would like to share my three R’s for participatory evaluation:
1. Build Rapport: In addition to conducting formal interviews and assessments, I interacted informally with the students and mentors when time allowed, during meals and in between activities. I spent time learning about Native American history and culture from the project team and students.
2. Demonstrate Relevance: I discussed with the stakeholders and participants possible benefits of the evaluation process and their unique roles in the improvement and success of the program components. For example, when the students expressed interest in helping future GEMscholars, a peer-mentoring option was added to the program. Consequently, students began to see the evaluation process as a mechanism for sharing their experiences and suggestions instead of an outside critique of their lives and activities.
3. Maintain Responsiveness: I provided the stakeholders with information in a timely and accessible format. Often these were oral reports followed by brief documents outlining the changes discussed. We had conversations about those issues that could not be resolved in a timely matter and possible effects on the program. In turn, the project team made ongoing changes, adding components where needed and modifying those elements that were not serving the objectives of the program. Assessments were modified if needed and the process continued.
Hot Tip: Journaling is a useful technique to capture real time reactions to interventions. This is particularly important when working with groups who are being introduced to unfamiliar and/or uncomfortable experiences as part of an intervention. I worked closely with the researcher and program coordinator to develop pertinent guiding questions for the students’ and mentors’ daily reflection journals. This is also a good time to develop an analysis rubric if applicable. Journals can be hand written or online (I provide a link to an online journal using Qualtrics). The journal entries provide a project team with valuable insights about how the program elements are perceived by all involved.
If you want to learn more from Susan, check out the Poster Exhibition on the program for Evaluation 2010, November 10-13 in San Antonio.
Hi Susan, the three R’s seem to make allot of sense when dealing with any issue. I used to do work with internal audits and this advice is good in that field as well.
The last thing people want to hear is the word audit, however if you take the time to build rapport, show the relevance of the audit and maintain relevance, it can actually turn into a productive endeavor.
The relevance in my mind is very important as if you can show the participant how the audit can benefit them in the future by showing where they need to focus, you can make the task at hand much easier.
Being responsive to the customer is extremely important as well. Your name and your reputation will get around as you most likely are in a specialized field. This can help or hamper you going forward.