This is a post in the series commemorating pioneering evaluation publications in conjunction with Memorial Day in the USA (May 28).
My name is Sharon Rallis, a former AEA President and editor of the American Journal of Evaluation. Carol Weiss was a pioneering sociologist and program evaluator who helped create the field of evaluation. She was my advisor and teacher, and taught me how evaluations can be used “to improve policy and programming for the well-being of all” (1998, p.ix).
Carol Weiss believed that understanding and using evaluation means integrating theory with practice, a perspective exemplified in the 1995 article she wrote for the Aspen Institute about the importance of basing evaluations on solid theories of change that underlie interventions. This article, “Nothing as practical as a good theory: Exploring theory-based evaluation for comprehensive community initiatives?for children and families”, became a classic. Today we would say: it went viral. Both the article – and the phrase Nothing as practical as a good theory — remains one of the most influential, if not the most influential, in the history of program evaluation. The influence can be found in that virtually every philanthropic foundation, major government agency, nonprofit, and international development organization requires that a theory of change be included in funding proposals and development initiatives.
Carol was reacting to millions of dollars being poured into community change efforts with little recognition of contextual complexities. Theory-based evaluation asks program practitioners to make their assumptions explicit and to reach consensus with their colleagues about what they are trying to do and why. While difficult, these conversations help practitioners reach shared understandings and offer evaluators insight into the “leaps of faith” (p. 72) embedded in their formulations of programs. She wasn’t just suggesting that a bunch of program people get together to share ignorance and biases, and fabricate a theory of change out of thin air, though that’s often what happens; rather, she proposed that they grapple with how their intervention, that is, what they do, connects with intended outcomes. Weiss reported her experience that “Program developers with whom I have worked sometimes find this exercise as valuable a contribution to their thinking as the results of the actual evaluation. They find that it helps them re-think their practices and over time leads to greater focus and concentration of program energies” (p72).
Lessons Learned:
Evaluations that address the theoretical assumptions embedded in programs may have more influence on both policy and popular opinion. ?According to Carol, “theories represent the stories that people tell about how problems arise and how they can be solved” (p. 72). We all have stories about the causes of and solutions to social problems, and these stories – or theories – accurate or not, play powerful roles in policy discussion. “Policies that seem to violate the assumptions of prevailing stories will receive little support” (72). It follows that evaluations grounded in clear and shared theories of change can inform and influence policy discourse.
To summarize, Carol Weiss wrote: “Grounding evaluation in theories of change takes for granted that social programs are based on explicit or implicit theories about how and why the program will work. The evaluation should surface those theories and lay then out in as fine detail as possible, identifying all the assumptions and sub-assumptions built into the program” (1995, 66-67). The insights she brought in her 11 published books and numerous journal articles have shaped how we think about and practice evaluation today.
Rad Resources:
Weiss, C.H. (1995). Nothing as practical as good theory. Washington, DC: Aspen Institute.
Weiss, C.H. (1998). Evaluation: Methods for Studying Programs and Policies 2nd Edition. Prentice Hall
Weiss, C.H. (1998). Have We Learned Anything New About the Use of Evaluation? American Journal of Evaluation,19: 21-33.
The American Evaluation Association is celebrating Memorial Week in Evaluation. The contributions this week are remembrances of pioneering and classic evaluation publications. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Dear Sharon,
My name is Curtis Thompson. I am a student at Queens University and am currently enrolled in a program evaluation course. I have been assigned the task of finding an article on AEA365 that interest me and writing a thoughtful response to the author of that article. My search for an article of interest was peaked when I happened upon your article, ‘The classic nothing as practical as good theory by Carol H. Weiss by Sharon Rallis’.
As I read through the article I kept finding myself nodding my head in agreeance and thinking that statement is so true. The statement regarding theory-based evaluation is so powerful, and I quote, “Theory-based evaluation asks program practitioners to make their assumptions explicit and to reach consensus with their colleagues about what they are trying to do and why.” The idea of making the design, implementation, and evaluation of a program into a collaborative experience that draws public attention to what the intentions of the program are and why the program is trying to achieve certain results is the most powerful tool in program design and evaluation. This concept is what makes a program sustainable and engaging. Why do you think most program designers leave certain key features about a program closed to many stakeholders? Do you think it is to garner influence over the program and what the program has to offer?
I was also drawn in when you quoted a statement by Carol Weiss stating that, “She wasn’t just suggesting that a bunch of program people get together to share ignorance and biases, and fabricate a theory of change out of thin air, though that’s often what happens; rather, she proposed that they grapple with how their intervention, that is, what they do, connects with intended outcomes”. In my experience in schools, the statement regarding a bunch of people that are of similar mind come together sharing some form of ignorance and bias piece together a “program” to elicit change happens more often than not. Colleagues are often forced to look through outside data and formulate a plan for change that will improve outcome performance. This type of forced collaboration often achieves the foreseeable outcome, creating a program plan, but is often left at that stage of development. Colleagues become collaborators for a day and do not use the program evaluation plan. What usable solution(s) have you witnessed or used in practice that better engages or re-energizes the collaborating team?
In lessons learned you concluded by stating Carol Weiss’ writing stating that, “Grounding evaluation in theories of change takes for granted that social programs are based on explicit or implicit theories about how and why the program will work. The evaluation should surface those theories and lay them out in as fine detail as possible, identifying all the assumptions and sub-assumptions built into the program”. It is without a doubt that Carol was groundbreaking in the field of evaluation. I think Carol would be of the school of thought that members of the evaluation team have to be truly passionate about the program they are designing, implementing and evaluating. Once an engaged focal group has been developed they can then start the process of program design and evaluation.
Thank you for sharing the theories of Carol H. Weiss. I truly appreciated your article and found it engaging and fun to read. I look forward to hearing back from you and will continue to read articles posted by you in the future.
Curtis Thompson
Thanks for sharing this! Also, as a social psychologist turned evaluator I just wanted to point out that it was Kurt Lewin who first said that “there is nothing as practical as a good theory” in the 1940s. Weiss acknowledges this in the notes for her paper and I love how it shows the intersections of psychology and evaluation. Thanks again!