This is part of a series remembering and honoring evaluation pioneers in conjunction with Memorial Day in the USA on May 30.
My name is Sharon Rallis, a former AEA President and current editor of the American Journal of Evaluation. Carol Weiss was my advisor and teacher; she taught me how evaluation can be used to make a better world. She said, “With the information and insight that evaluation brings, organizations and societies will be better able to improve policy and programming for the well-being of all” (Weiss, 1998, p.ix). Her 11 published books and numerous journal articles shaped how we think about evaluation today.
Pioneering and enduring contributions:
Carol’s visionary contributions began in the 1960s with research on evaluation use. Her book Evaluating Action Programs (1972) pioneered utilization as a field of inquiry. She was among the first to recognize the importance of program context as well as roles evaluators play in use – and that the use might not be what was expected. She illuminated the politics of evaluation: programs are products of politics; evaluation is political; reports have political consequences; politics affect use. Carol once told me that “decision makers are human; they filter data through their beliefs, values, their agendas and ideologies. How – and whether – they use the information depends on how you communicate – can you make the information relevant? After all, you probably won’t even see them use it – there may just be a shift in the way they think.” In sum, she expanded our views of use from instrumental to incremental or enlightenment.
Carol evaluated and reflected on what and how she had evaluated, connecting theory and practice. In her classic Nothing as Practical as Good Theory, she wrote: “Grounding evaluation in theories of change takes for granted that social programs are based on explicit or implicit theories about how and why the program will work. The evaluation should surface those theories and lay then out in as fine detail as possible, identifying all the assumptions and sub-assumptions built into the program” (1995, 66-67). Her argument shapes how many of us work with the decision makers in programs we evaluate.
Finally, she had a wonderful sense of humor. Her titles include intriguing phrases like: “Treeful of Owls”; “The fairy godmother and her warts”; and “What to do until the random assigner arrives”. She filled her conversations with everyday insights and ordinary reasons to laugh. Carol humanized evaluation.
Resources:
Weiss, C.H. (1998). Evaluation: Methods for Studying Programs and Policies 2nd Edition. Prentice Hall
Weiss, C.H. (1998). Have We Learned Anything New About the Use of Evaluation? American Journal of Evaluation,19: 21-33.
The American Evaluation Association is celebrating Memorial Week in Evaluation: Remembering and Honoring Evaluation’s Pioneers. The contributions this week are remembrances of evaluation pioneers who made enduring contributions to our field. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
It is fun to find this post and reflect on Carol Weiss. If bands have fan clubs, evaluators have role models, inspirational leaders or favorite influencers. I wish I could have known her and have always been drawn to her focus on use. I think countless evaluators today benefit from her wisdom, writing and insights. I know I have.
Thank you.
Kate
Hi Sharon,
I came across your article while taking a course in Program Inquiry and Evaluation as I work toward my Professional Master of Education from Queen’s University. I was recently introduced to Dr. Weiss’s work while reading about dilemmas in evaluation use. Initially, I watched a video by Michael Q. Patton’s on Utilization-Focused Evaluation before reading Weiss’s “Have we learned anything new about the use of evaluation?” Her position on evaluation use immediately resonated with me, and the manner in which she addressed the “debate” with Patton made it apparent that she indeed did have a good sense of humour, as you mentioned!
You say that Carol once told you “…How – and whether – they use the information depends on how you communicate.” As someone who is new to the field of evaluation, I was struck by how her writing is accessible not only to her peers but to newbies like myself as well. I expect that her approachable writing style, which can be so rare in academia, played a significant role in ensuring that evaluators write evaluations that are accessible and coherent to the intended users (so that they acutally get used).
Sharon, thank you for introducing me (posthumously) to the person Carol was and the impact she had (and will continue to have) on the evaluation community. I will look forward to reading the additional resources you cite, as I won’t be able to resist reading the articles with the hilarious titles!
Kindest Regards, Jen.
Hi Sharon,
It is absolutely amazing that Carol Weiss was your advisor and teacher. What an incredible opportunity to learn and work alongside someone who was such a prominent leader in the field of evaluation. She has truly left an everlasting impact on the world of evaluation.
You quoted, “With the information and insight that evaluation brings, organizations and societies will be better able to improve policy and programming for the well-being of all”. When I read this quote, I couldn’t help but reflect on an article written by Weiss and Rogers (2007) that reflected on theory evaluation and the challenges associated with it. The article points out that most theory-based evaluations do not provide enough useful information to conduct effective program evaluations and create positive change and direction meaningfully. Further, Weiss clearly states how she disagrees with how program theory is often used for evaluation. She explains that too often a program is developed but then it is not used to guide the evaluation itself.
As you pointed out, many of Weiss’ arguments about evaluation shape how many of us work and frame evaluations. Her impact on the field is huge and will always be foundational.
Her argument shapes how many of us work with the decision makers in programs we evaluate. Evaluation is very political and drives policy making.
Because of this, it is important that evaluators are trained well regarding evaluation. Luckily, you were able to learn from one of the best.
Thank you for sharing this wonderful article and highlighting her pioneering contributions to the field.
Sincerely,
Jenna
Rogers, P. J., & Weiss, C. H. (2007). Theory?based evaluation: Reflections ten years on: Theory?based evaluation: Past, present, and future. New Directions for Evaluation, 114: 63-81.