EERS Week: Evaluating in the Midst of Complexity by Patricia Moore Shaffer

Hi, I’m Patricia Moore Shaffer, Deputy Director, Research & Analysis, at the National Endowment for the Arts and Board Member of the Eastern Evaluation Research Society. The National Endowment for the Arts is the independent federal agency whose funding and support gives Americans the opportunity to participate in the arts, exercise their imaginations, and develop their creative capacities. As the lead program evaluator, I often work with the challenge of evaluating complex initiatives in which arts organizations partner with other community-based organizations, local government agencies, and the philanthropic sector to improve communities. I’m delighted to share a few tips and resources on approaching the study of complex initiatives.

Program or project evaluations often rely on a linear theory of change, but a single theory may be insufficient for cross-sectoral community change initiatives. These complex initiatives may draw upon multiple theories or frameworks from several disciplines (e.g., education, biology, economics) and may benefit from the use of nested or linked theories of change that break down program theory into digestible chunks. Systems maps or webs may be a more appropriate way to visualize a complex initiative than a linear theory of change or logic model.

Evaluations of complex initiatives succeed using an approach that respects that community change is inherently non-linear, highly iterative, and contextual in nature. Developmental evaluation, which focuses on adaptive learning, providing real-time feedback to support the initiative’s development, and understanding how complex initiatives work, is particularly well-suited for this evaluand.

Rad Resources:

I found a few useful resources on my path to evaluating in the midst of complexity:

The American Evaluation Association is celebrating Eastern Evaluation Research Society (EERS) Affiliate Week. The contributions all this week to aea365 come from EERS members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.