This is a post in the series commemorating pioneering evaluation publications in conjunction with Memorial Day in the USA (May 28).
My name is Sharon Rallis, a former AEA President and editor of the American Journal of Evaluation. Carol Weiss was a pioneering sociologist and program evaluator who helped create the field of evaluation. She was my advisor and teacher, and taught me how evaluations can be used “to improve policy and programming for the well-being of all” (1998, p.ix).
Carol Weiss believed that understanding and using evaluation means integrating theory with practice, a perspective exemplified in the 1995 article she wrote for the Aspen Institute about the importance of basing evaluations on solid theories of change that underlie interventions. This article, “Nothing as practical as a good theory: Exploring theory-based evaluation for comprehensive community initiatives?for children and families”, became a classic. Today we would say: it went viral. Both the article – and the phrase Nothing as practical as a good theory — remains one of the most influential, if not the most influential, in the history of program evaluation. The influence can be found in that virtually every philanthropic foundation, major government agency, nonprofit, and international development organization requires that a theory of change be included in funding proposals and development initiatives.
Carol was reacting to millions of dollars being poured into community change efforts with little recognition of contextual complexities. Theory-based evaluation asks program practitioners to make their assumptions explicit and to reach consensus with their colleagues about what they are trying to do and why. While difficult, these conversations help practitioners reach shared understandings and offer evaluators insight into the “leaps of faith” (p. 72) embedded in their formulations of programs. She wasn’t just suggesting that a bunch of program people get together to share ignorance and biases, and fabricate a theory of change out of thin air, though that’s often what happens; rather, she proposed that they grapple with how their intervention, that is, what they do, connects with intended outcomes. Weiss reported her experience that “Program developers with whom I have worked sometimes find this exercise as valuable a contribution to their thinking as the results of the actual evaluation. They find that it helps them re-think their practices and over time leads to greater focus and concentration of program energies” (p72).
Evaluations that address the theoretical assumptions embedded in programs may have more influence on both policy and popular opinion. ?According to Carol, “theories represent the stories that people tell about how problems arise and how they can be solved” (p. 72). We all have stories about the causes of and solutions to social problems, and these stories – or theories – accurate or not, play powerful roles in policy discussion. “Policies that seem to violate the assumptions of prevailing stories will receive little support” (72). It follows that evaluations grounded in clear and shared theories of change can inform and influence policy discourse.
To summarize, Carol Weiss wrote: “Grounding evaluation in theories of change takes for granted that social programs are based on explicit or implicit theories about how and why the program will work. The evaluation should surface those theories and lay then out in as fine detail as possible, identifying all the assumptions and sub-assumptions built into the program” (1995, 66-67). The insights she brought in her 11 published books and numerous journal articles have shaped how we think about and practice evaluation today.
Weiss, C.H. (1995). Nothing as practical as good theory. Washington, DC: Aspen Institute.
Weiss, C.H. (1998). Evaluation: Methods for Studying Programs and Policies 2nd Edition. Prentice Hall
Weiss, C.H. (1998). Have We Learned Anything New About the Use of Evaluation? American Journal of Evaluation,19: 21-33.
The American Evaluation Association is celebrating Memorial Week in Evaluation. The contributions this week are remembrances of pioneering and classic evaluation publications. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to firstname.lastname@example.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.