Hi, we are Wanda D. Casillas, Ph.D., University of Michigan, and Asma M. Ali from the American Society for Clinical Pathology in Chicago, IL. We have been thinking about interaction of culturally responsive evaluation and traditional research in our own evaluation work since we met as AEA-Duquesne GEDI Scholars in 2008-2009.
In particular, we have been thinking about the question “Does traditional social science research have a place in how we practice CRE as evaluators and in how we systematically determine ‘best CRE practices’?” Among the considerations that have informed our discourse are:
- How does culture factor into evaluation? Within the evaluation field, we are grappling with issues of defining culture for an evaluation context, understanding why culture matters to our work, and what constitutes culturally responsive practices both domestically within the U.S. and on the world stage. As evaluators, our facilitative roles in global and domestic evaluations must be informed by an understanding of culture in order to promote program success and conduct valid evaluations.
- What is the role of research in culturally responsive evaluation practice? For the most part, evaluators have integrated research-like practices into evaluation approaches and argued only subtle points of distinction. However, some culturally responsive evaluators have been less compromising. CRE theorists and practitioners urge professionals to challenge traditional methods of data collection (borrowed from social science research) and argue that traditional research approaches privilege the types of questions asked, data collected, measures used, and interpretation of data. As a result, being culturally responsive requires flexibility in considering what constitutes valid forms of evidence in evaluations.
- What constitutes evidence in a culturally responsive evaluation? In our modern evaluation climate we see an increasing demand for evidenced-based practice, and evaluators are often charged with gathering the “evidence” in EBP. Recently evaluators have begun to question what exactly constitutes “evidence” in an evaluation context. Disentangling this question means that evaluation professionals must look toward our practices which are often rooted in the social sciences and question the appropriateness of these practices for the evaluation contexts in which we work. For example, is a randomized control trial always the best possible design for an evaluation, as many government agencies are now assuming and demanding?
Lesson Learned: We have come to realize that these questions are important when considering the relevance of culturally responsive evaluation in broader evaluation discourse. What do you think- Should we use traditional research methods to test and validate any testable CRE practices in ways that help build a protocol for CRE? What are the broader implications of using traditional research methods in this manner?
This week, we’re diving into issues of Cultural Competence in Evaluation with AEA’s Statement on Cultural Competence in Evaluation Dissemination Working Group. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.