Greetings! This is Darla Scott and I participated as an AEA 2018 MSI Fellow. I also enjoy serving as an Assistant Professor in the School Psychology program at Bowie State University. At Bowie State University, we maintain a specialist-level three year NASP-approved program that focuses on creating culturally competent school psychologists, and I have the privilege of teaching the research courses in our program.
During Evaluation 2018, the MSI Fellows presented, “Teaching Culturally Responsive Evaluation: Theory, Practice, and Reflection”, which focused on the curricular integration of culturally responsive evaluation and CRE learner experiences. I contributed by examining participant reflections on learning about culturally responsive evaluation. This qualitative data essentially highlighted the importance of project-based, culture-laden activities with real-world examples and applications. It also emphasized that novice learners experience less anxiety and confusion than more professional learners with limited prior exposure to CRE. Based upon this feedback and as a final project for my MSI Fellowship, I designed hands on learning exercises to expand my current program evaluation module in our Research Methods and Statistics in School Psychology course. I will share a few course enhancements used to redesign this course.
For the program evaluation module, scholars independently devise an evaluation plan for an actual educational program. To enhance this unit, more details about the community demographics and culture were included about the assigned program. Further, the students were required to plan specific community engagement events to involve the stakeholders (both the powerful and potentially marginalized groups) in the evaluation planning process. Finally, I developed two role-play exercises with reflection for this module: (1) a mock community engagement event; and (2) the stakeholder evaluation presentation. For the mock community engagement event, students take different “roles” such as students, teachers, or parents, and pose questions or make suggestions for the design of the evaluation from that lens. For the stakeholder evaluation presentation, the scholars are required to present their logic model and evaluation plan as the “evaluator” to the “stakeholders” that are enacted by their classmates who are tasked with listening and posing questions with an assigned perspective. Both of these role-play exercises conclude with a time of discussion, reflection, and feedback. These course enhancements were piloted and well-received by my students during the fall.
Rad Resources:
Great summary of the history and conceptualization of CRE – Hood, Hopson, & Kirkhart (2015). Culturally responsive evaluation.
Great framework for approaching the teaching and learning of CRE – Boyce & Chouinard, (2017). Moving beyond the buzzword: A framework for teaching culturally responsive approaches to evaluation.
Excellent instructional resource for teaching CRE – Bowen & Tillman, (2015). Developing culturally responsive surveys: Lessons in development, implementation, and analysis from Brazil’s African descent communities.
Practical Example for teaching about CRE – Boyce (2017). Lessons learned using a values-engaged approach to attend to culture, diversity, and equity in a STEM program evaluation.
The American Evaluation Association is AEA Minority Serving Institution (MSI) Fellowship Experience week. The contributions all this week to aea365 come from AEA’s MSI Fellows. For more information on the MSI fellowship, see this webpage: http://www.eval.org/p/cm/ld/fid=230 Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hi Darla,
My name is Yajing. I am currently taking the course-Program Inquiry and Evaluation at Queens University. Your article interests me a lot.
Nowadays, with more and more focus on cultural diversity, culturally responsive evaluation is an essential topic to be explored and discussed. It is important to consider cultural context when doing project evaluation. However, transferring culturally responsive theory to evaluation practice is not always an easy task. I agree with your idea that “I was also emphasized that novice learners experience less anxiety and confusion than more professional learners with limited prior exposure to CRE”. It is reasonable that professional learners with more experience and knowledge are harder to apply the CRE without any bias. I have a good example of this based on our daily experience. When I was preparing my driver license road test, I was nervous that it would be hard for me to learn it because I didn’t have many driving experiences before. However, my driving coach said I am easier to teach than those old drivers because I didn’t have bad driving habits. Novice learners without many evaluation experience and knowledge are easier to accept the CRE and apply it into evaluation practice.
The mock community engagement event and the stakeholder evaluation presentation you implemented in the class module are amazing! By playing different roles of the community, students are more able to consider the different cultural background of the roles they are playing into the evaluation process. Stakeholder evaluation presentation gives students the chance to listen to others and engage the perspectives of others.
Your process of teaching students culturally responsive evaluation looks fabulous and practical. I am curious is there any challenge that you encounter when you applying and teaching CRE?
Thank you for an interesting post!
Yajing
Thank you for your feedback! Yes, the main challenges center around students feeling comfortable taking “other” perspectives. It can be challenging for my students to take ownership and fully commit to those roles. Their anecdotal reactions have been positive, but the process takes time for the students to acclimate. I apologize for the delay in my reply and I appreciate your question!
Darla
Hi Darla
I was very intrigued by your recent submission to the AEA365 blog regarding CRE. The one quote that especially stuck me was “This qualitative data essentially highlighted the importance of project-based, culture-laden activities with real-world examples and applications. It also emphasized that novice learners experience less anxiety and confusion than more professional learners with limited prior exposure to CRE.”
The first thought that came to mind was that you can’t teach an old dog new tricks, but I think that is exactly the mindset that you are striving to fight against. Students today (specifically in Ontario, where I am an educator) are well familiar with a variety of assessment tools and the integration of cultural examples. It is part of our “Growing Success” document supplied by the Ministry of Education. This is vastly different for the academic landscape of 10-15 years ago. I can well see how learners with more experience would be uncomfortable with this type of teaching and assessment.
I was very excited to see a role playing exercise as part of the process toward helping others feel comfortable with the evaluation process. We have been learning about different evaluation theories and methods in my Professional Master’s of Education course and it highlights the theories that evaluators and stakeholders should have interaction and a good understanding of cultural context for best results. Thanks for a great article.
Regards,
Nicole Brouwer
Thank you for your thoughts and comments!
Darla
To further my thoughts…
The engagement event activity is such a great way to create empathy in people and have them consider the perspectives of minority groups to which they likely do not belong. I could also see it fostering a deeper commitment to those students of minority backgrounds. The evaluation presentation would, yet again, be a great way to engage the perspectives of others and consider varying viewpoints. Have you had any instances where this created a debate amongst what would be best for the varied demographic groups? I’m curious if some students disagreed with how potential culturally sensitive issues could or should be handled.
Thanks again Darla for some great information!
Melissa
Hello Melissa!
Thank you for your kind words and thoughts! May I recommend an online simulation called Spent (http://playspent.org/)? This is a very engaging web based exercise on perspective taking that could work for educators. One of my students found it and we used it with my graduate students.
My in-class debates have not reached that depth yet… but I am optimistic that we will get that point as we dig into the issues. You are making me think of ways to get the students more personally invested in the issues… one idea is to have my students find some relevant research articles to support the development of their informed opinion on the focal topic.
Darla
Hi Darla, this is Melissa an Albertan educator and administrator currently enrolled at Queen’s University in their Professional Master of Education program specializing in evaluation and assessment.
Your article and work really resonated with me as I work in an ethnically diverse school so I found myself nodding at many of your comments especially those regarding the importance of “…project-based, culture-laden activities with real-world examples and applications”. I think this is so very important for schools and teachers to understand especially those will diverse minority populations. I loved the model you discussed and found myself wondering how I could incorporate something like that in a professional development session for my school district. I think sharing the demographic and cultural details about the various minority groups and moving into that mindset could really help teachers grow in their understanding of the challenges faced by their students and families.
As an administrator, I am moved and placed into a new school every few years so your article has provided me with some new ideas and tools for my current and prospective schools so thank you for sharing your wisdom!
Take care,
Melissa