OCF Week: Kim Leonard on Studio to School – A developmental evaluation in action

I am Kim Leonard, Senior Evaluation Officer at The Oregon Community Foundation. Today I want to share lessons learned from a developmental evaluation we’re undertaking for our five-year arts education grantmaking initiative – Studio to School.

The nature of this Initiative is developmental and rooted in the arts. Creativity, adaptation, and risk are supported. The first phase of the evaluation is focused on understanding and supporting the arts education programming being developed by the project teams funded through Studio to School. We are now approaching the mid-point of the Studio to School Initiative evaluation, and have learned a lot about the benefits and challenges of implementing a developmental evaluation.

Lesson Learned: Taking a developmental evaluation approach has allowed the Research team to adapt the evaluation in response to the evolution of the Initiative. It took us a little while to get used to this approach! We’ve summarized our evaluation on this handout, and find ourselves coming back to it repeatedly to keep us grounded as we plan new evaluation activities.

Lesson Learned: The Research team has worked in an ongoing way to develop rigorous evaluation activities to collect and provide useful information in a feedback loop. Robust reflection is built into the process; debrief meetings are held following each major learning community and evaluation activity to share and document learnings. These often turn into planning sessions for future evaluation and learning community efforts. In addition, the project teams are journaling electronically – quarterly reflections on what they are learning in response to prompts have been one of the most valuable data sources to date. Prompts (like this example) are developed one or two at a time, so that they are as timely and relevant as possible.

Lesson Learned: A key element of the evaluation, and goal of the Initiative, is to surface and articulate principles of high quality sustainable arts education programming. We began developing principles based on the first year’s evaluation findings, and asked project teams to reflect and provide feedback on draft principles at a recent gathering. We were thrilled with how engaged the teams were in this effort. The photo below shows a project team member reviewing feedback provided on sticky notes. Attendees also placed red dots (as seen in photo) next to those principles that most resonated with their experience. Doing this as a larger group allowed project teams to discuss their feedback and for attendees to react to and comment on one another’s feedback.

Rad Resources: In addition to the excellent Developmental Evaluation Exemplars (Patton, McKegg, and Wehipeihana, 2016), we have found the Developmental Evaluation Primer and DE 201: A Practitioner’s Guide to Developmental Evaluation from the McConnell Foundation especially helpful. Additional resources are listed at http://betterevaluation.org/plan/approach/developmental_evaluation.

The American Evaluation Association is celebrating Oregon Community Foundation (OCF) week. The contributions all this week to aea365 come from OCF team members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.