Hi, I’m Patricia Moore Shaffer, Deputy Director, Research & Analysis, at the National Endowment for the Arts and Board Member of the Eastern Evaluation Research Society. The National Endowment for the Arts is the independent federal agency whose funding and support gives Americans the opportunity to participate in the arts, exercise their imaginations, and develop their creative capacities. As the lead program evaluator, I often work with the challenge of evaluating complex initiatives in which arts organizations partner with other community-based organizations, local government agencies, and the philanthropic sector to improve communities. I’m delighted to share a few tips and resources on approaching the study of complex initiatives.
Program or project evaluations often rely on a linear theory of change, but a single theory may be insufficient for cross-sectoral community change initiatives. These complex initiatives may draw upon multiple theories or frameworks from several disciplines (e.g., education, biology, economics) and may benefit from the use of nested or linked theories of change that break down program theory into digestible chunks. Systems maps or webs may be a more appropriate way to visualize a complex initiative than a linear theory of change or logic model.
Evaluations of complex initiatives succeed using an approach that respects that community change is inherently non-linear, highly iterative, and contextual in nature. Developmental evaluation, which focuses on adaptive learning, providing real-time feedback to support the initiative’s development, and understanding how complex initiatives work, is particularly well-suited for this evaluand.
Rad Resources:
I found a few useful resources on my path to evaluating in the midst of complexity:
- Developmental Evaluation offers an approach to evaluating change initiatives in complex environments, and the best guide to this approach is Michael Quinn Patton’s Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use.
- Evaluating Complexity: Propositions for Improving Practice, written by Hallie Preskill and Srik Gopal, offers a series of propositions to evolve the practice of evaluation of complex initiatives, as well as initiatives operating in complex environments.
- John Mayne’s Useful Theory of Change Models offers a thorough discussion of approaches to theories of change, including developing theory for “messy interventions.”
The American Evaluation Association is celebrating Eastern Evaluation Research Society (EERS) Affiliate Week. The contributions all this week to aea365 come from EERS members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.