We are Michael Quinn Patton, founder and original author of Utilization-Focused Evaluation, and Charmagne E. Campbell-Patton, Director of Organizational Development and Evaluation for our Utilization-Focused Evaluation consulting practice. We are father-daughter and evaluation partners.
A major challenge in designing and carrying out evaluations in these turbulent times is dealing with complexity. In working with intended users to consider these distinctions, it is illuminating to engage them in discussion about what aspects of what they do are relatively simple, relatively complicated, and relatively complex from their perspectives. Questions and designs can then be matched to these distinctions.
We face the same challenge in writing about utilization-focused evaluation, presenting the process as a series of sequential steps while also capturing the complex nature of the utilization-focused process as non-linear, interactive, dynamic, and adaptive. The utilization process is not neatly linear and sequential. There are interactions among the steps, feedback loops, recursive iterations, interrelationships, and interdependencies among those involved in the evaluation. For example, while identifying primary intended users occurs at the beginning of a utilization-focused evaluation, that process may be ongoing (Do we have the right intended users as evaluation questions emerge?) and new intended users have to be identified when there is turnover among those originally involved. In essence, the utilization-focused evaluator is asked to operate within two perspectives at the same time, one simple, stepwise and neatly linear, the other complexly interactive and dynamic. To manage this complexity, we have found it useful to be explicit about and follow the fundamental principles of utilization-focused evaluation, so that is how we have organized the new 5th edition of the book just published and launched as part of this year’s American Evaluation Association conference.
Intergenerational Engagement
The new edition integrates Michael’s 50 years’ experience with Charmagne’s perspective as a second generation evaluator, bringing attention to emergent trends and issues with new questions and perspectives. Together we are committed to engaging in evaluation in ways that contribute to a more just and sustainable future. That has led to a complete rewrite and revision of the book.
Evolution of Utilization-Focused Evaluation
The 1st edition published in 1978 featured early research on factors consistently affecting use: (1) political considerations and (2) “the personal factor” — the presence of an identifiable individual or group of people who personally care about the evaluation and the findings it generates. Where such a person or group was present, evaluations were used; where the personal factor was absent, there was a correspondingly marked absence of evaluation impact.
The 2nd edition (1986) offered for the first time a definition of utilization: “intended use by intended users.” This focus proved to have implications for everything done in an evaluation and every decision made along the way. Working with primary intended users to enhance intended uses became the hallmark of utilization-focused evaluation.
The 3rd edition (1997) introduced process use. Up until that time the entire field had narrowly focused on use of findings. This idea of process use draws our attention to individual changes in thinking and behavior among those involved in the evaluation as a result of the learning that occurs during the evaluation process.
The 4th edition (2008) gave significantly increased attention to international and cross-cultural factors that affect use and brought systems thinking and complexity theory into utilization-focused evaluation.
The Ongoing Evolution of Utilization-Focused Evaluation
This new 5th edition for the first time presents and elaborates principles of utilization-focused evaluation. Principles guide application and adaptation. Principles connect theory with practice, questions with methods, and findings with follow-through actions. The 21st century has brought major changes in the world and, correspondingly, in the evaluation profession, like the global adoption of Sustainable Development Goals for Agenda 2030 that expresses a commitment to equity and sustainability. This new edition presents for the first time the role of utilization-focused evaluation in addressing issues of equity and sustainability in the context of the COVID-19 pandemic, the climate crisis, and the global social justice uprising. As evaluators, we have a stake in having our evaluations used—and we have a stake in a more equitable and sustainable world. The implications for utilization-focused evaluation are substantial, dramatic, controversial, and forward-looking.
Rad resources
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.
Hello Micheal and Charmagne,
Thank you for sharing a very succinct description of my journey with program evaluation thus far. I am a graduate student currently taking a course in program evaluation; in the beginning, my deep dive into the field felt like I was drowning in a sea of terms, concepts, and structures that seemed to defy my understanding. Specifically, as you mentioned, I could not wrap my head around the complexity of synthesizing linear steps with the interactive iterative aspects between the steps. As you state, the “evaluator is asked to operate within two perspectives at the same time, one simple, stepwise, and neatly linear, the other complexly interactive and dynamic”; but, ironically, it was when I learned about utilization-focused evaluation that I began to understand the complex relationship between the two perspectives. Utilization-focused evaluation just made sense to me and this approach helped unlock the mysteries of program evaluation as a whole. Approaching evaluation with the goal of intended use for intended users in a context rich collaborative environment resonated with my pedagogical approach as a classroom teacher. I began to see program evaluation through the context of my own experience working with students and evaluation. Furthermore, I’m excited to see that the evolution of Utilization-focused evaluation is committed to considering and incorporating current issues and trends within the process. It proves that the very framework of utilization-focused evaluation adheres to its own principles of being dynamic and of revision, reflection, collaboration, interaction.
Thank you,
Mona Haidar
Hello Michael and Charmagne,
I have been taking a course on program evaluation through Queens University for my Masters in Education. I did not know a lot about evaluation prior to taking this course and my understandings on the importance of evaluation has grown through reading various articles on this AEA365 blog.
I enjoyed reading your article, and I believe it does a great job at showcasing how complex an Evaluator’s role is and how the role on an evaluator has changed over time. Over the last several weeks in my course, I have been wondering how a utilization-focused evaluator is able to operate in such a complex role? I appreciated your statement in that it is, “useful to be explicit about and follow the fundamental principles of utilization-focused evaluation.” I look forward to exploring your book to see the suggestions that you make and how the use of these 5 principles are explained further.
Thank you for your words,
L. Shirley
Dear Michael and Charmagne,
I am currently registered in a course called Program Inquiry and Evaluation and I just wanted to write to say that I found your article intriguing. As someone new to Program Evaluation, I find the whole process to be quite complex. One of the first times that I truly felt like I understood the whole process was after watching one of Michael’s Youtube videos. Your newest edition of your book sounds interesting. Utilization-focused evaluation appeals to me because I do think there is an importance of follow-through and connection from the position of the evaluator.
I am a classroom teacher and if someone were to ask me how to teach a child to read, my answer probably wouldn’t make it any more clear because there are so many factors to consider, I feel like it must be similar for program evaluation design. As you state, there are “interactions among the steps, feedback loops, recursive iterations, interrelationships, and interdependencies.” In one way, addressing the complexity of the process makes it easier to understand.
Thank you for your article and for the work that you do in this area.
Celia Brouwer