My name is Lori Wingate, and I work at The Evaluation Center at Western Michigan University. I’ve been involved in a number of discussions around the issues of culture, color, and gender in evaluation. I value the premise from which these conversations start. Being hyper-pragmatic, I tend to get frustrated with where they end—that, is, without clear-cut answers about how to translate our heightened awareness into our practice as evaluators.
Professor Rodney Hopson spent three days at WMU, engaging the staff, students, faculty, and community members in conversations about culture and power in evaluation. His visit had a transformative effect on me. No, I did not come away with a credential in cultural competence (and you should be highly suspect of anyone who claims “cultural competence” without reference to a specific context). But I learned a valuable lesson that permeates my professional life.
Lesson Learned: It’s OK to not get to “the answer” on the big questions concerning the role of culture and power (or color, gender, etc.). There is value in the dialogue itself, and collectively such conversations move us forward as individual evaluators and as a discipline. In a profession focused on “measurable change,” it is easy to devalue incremental changes that happen in a less-than-systematic fashion. But small or “unmeasurable” experiences eventually lead to a better-informed evaluation practice; the accretion of tacit knowledge moves us toward wisdom. I still don’t know precisely what to do to ensure cultural competence in my work, but I have a better understanding of where my blind spots are.
Professor Hopson didn’t come to WMU with all the answers. Rather, he challenged us with questions. Consider the title of his lecture, “Evaluation & Public Good: Toward Whose Good, Whose Benefit, and What End?” These are questions we should continually ask ourselves as evaluators regarding our individual practice and our profession’s role within society. They hark back to Michael Scriven’s original two-fold definition of metaevaluation as the (1) “assessment of the role of evaluation” and (2) “evaluation of specific evaluative performances.” Although we tend to focus on the latter definition, it would serve as well to occasionally revisit the former and reflect on questions such as those posed by Professor Hopson—even if we end up with more questions than answers.
Rad Resource: Michael Scriven’s often-referenced, rarely read original article on metaevaluation. It’s a good reminder that metaevaluation isn’t just about evaluating individuals evaluations, but also the role of evaluation in society: Scriven, M. (1969). An introduction to meta-evaluation. Educational Products Report, 2, 36-38. (It’s not online, but I’d be happy to email a PDF upon request.)
Rad Resource: Video of Rodney Hopson’s WMU lecture, “Evaluation & Public Good,” accessible from http://www.wmich.edu/evalctr/2011/09/evaluation-and-the-public-good/
All this week, we’re highlighting posts from colleagues at Western Michigan University as they reflect on a recent visit from incoming AEA President Rodney Hopson. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hi Lori,
My name is Jordan and I’m a Master’s student at Queen’s University. Thanks for sharing your perspective on evaluation!
I think it’s really important to recognize that we don’t need to always find “the answer” when completing an evaluation. When I first started learning about evaluation in a professional setting, I felt a bit overwhelmed about the idea of answering extremely deep questions about serious topics. Some of the topics you alluded to such as issues of colour, race and gender are great examples of content areas that require deep levels of critical thinking and years of research. I’m curious about answering some of these questions myself, but I’ve since learned that it’s not always possible to answer them within a single evaluation.
One quote from your blog post that really stood out to me was when you said “there is value in the dialogue itself, and collectively such conversations move us forward as individual evaluators and as a discipline.” I completely agree with this sentiment. There is so much value in the process of evaluation itself, and much can be learned from it. Patton’s research has helped highlight that process use has several clear benefits, including leading to not only individual but organizational development (Shulha and Cousins, 1997). This falls in line with what you mentioned about the process of an evaluation itself helping you recognize your “blind spots,” thus making you a better evaluator moving forward.
A question I was wondering about was your reference to Professor Rodney Hopson. It sounds like he has been an extremely powerful influence on you as an evaluator which is fantastic. You provided a link to a lecture he gave at Western Michigan University as your Rad Resource, but unfortunately the link no longer works. Would you be able to provide a link to similar work of Professor Hopson’s that is similar?
Thank you for your time!
—-
References
Shuhla, L. M., & Cousins, J. B. (1997). Evaluation Use: Theory, Research and Practice Since 1986. Evaluation Practice, 18(3), 195–208.
I have received and responded to several requests for Scriven’s “An Introduction to Meta-Evaluation” article.
Here is another article I like for perspective on the “meta” part of metaevaluation:
Thomas, R. M. (1984). Mapping meta-territory. Educational Researcher, 13, 161-18.
If you are affiliated with a university, you probably have an institutional subscription to the journal–if so, this link should work:
http://edr.sagepub.com/content/13/1/16
Enjoy!