Sheila B Robinson on Creating a Culture of Evaluation

Hi! I’m Sheila B Robinson, aea365’s Lead Curator and sometimes Saturday contributor. I recently gave a keynote speech at a statewide conference for educators on how they can create a culture of evaluation in their school districts. As I looked for source material, I found a few Rad Resources I’d like to share with you today.

To anchor the audience in the importance of school culture on improvement, I opened with this quote from a book on educational leadership:

Numerous studies of school change have identified the organizational culture as critical to the successful improvement of teaching and learning. In study after study, where the culture did not support and encourage reform, the improvement did not occur (Deal, T.E., and Peterson, K.D., Shaping School Culture: The Heart of Leadership).

To begin creating a culture of evaluation, I encouraged the audience to do some “myth busting” –  to become familiar with common dysfunctional attitudes about evaluation and learn how we can address them in our organizations.

Rad Resource: “Dysfunctional attitudes toward program evaluation” in Program Evaluation: Methods and Case Studies by E.J. Posovac and R.G. Carey (7th edition). These authors do a wonderful job outlining nine “political and emotional factors that can undermine an evaluation project” (p. 40).

I then described what it looks like when an organizational culture supports and embraces evaluation for continuous improvement.

Rad Resource: Developing a culture of evaluation and research, a publication by Child Family Community Australia that provides “practical information on the structures, practices and actions that support a change toward a strong culture of evaluation and research.”

I then described what it doesn’t look like – that is, when an organization only goes partway, or cherry picks bits and pieces.

Rad Resource: Building an Evaluative Culture for Effective Evaluation and Results Management by John Mayne. This paper  describes characteristics of a culture of evaluation, but also articulates what happens in weaker cultures of evaluation, and lists disincentives for building an evaluative culture.

Finally, I explained the concept of evaluative thinking and how that neatly dovetails with creating a culture of evaluation.

Rad Resource: Tom Archibald and Jane Buckley introduced me to the concept of evaluative thinking with their aea365 post, Tom Archibald and Jane Buckley on Evaluative Thinking: The Je Ne Sais Quoi of Evaluation Capacity Building and Evaluation Practice as a well as an excellent Coffee Break Webinar on the topic. Tom continues writing about ET in his blog, free-range evaluation: reflections on evaluative thinking and evaluative doing.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.