Hi! My name is Patricia Moore Shaffer, and I am the Deputy Director, Research & Analysis, at the National Endowment for the Arts, in Washington, DC.
Theory of change and logic modeling are tools that program managers and evaluators often use to describe how a program is supposed to work. A theory of change permits the reader to understand the underlying hypotheses of a program by illustrating the program’s outcomes pathway – the activities understood to produce a series of results that contribute to achieving the desired change. A logic model, meanwhile, gives more clarity about the components (i.e., inputs) that need to be in place for the program to work; it also includes visual depictions of how the planned activities will result in specific outputs or products, and outcomes.
As an internal evaluator in a small federal agency, I often use theory of change and logic modeling to help program stakeholders articulate how their program works. It is helpful to engage a broad cross-section of stakeholders, including staff, beneficiaries, partners, and funders, in the development of these models. In addition to producing valuable tools to guide program design and development, stakeholders participating in this process will increase their knowledge and understanding of the program.
Hot Tip: Developing a theory of change or logic model for a program is a powerful learning process, and evaluators working collaboratively with stakeholders should keep in mind principles of adult learning when planning theory of change or logic modeling sessions:
- Adults want or need to be involved in how our learning experiences are planned and delivered. Consult with stakeholders about how to organize the model development work. Do they want to organize a day-long retreat or a series of short meetings with interspersed independent group work?
- Adults draw upon our own knowledge and experience when learning. Stakeholders have rich knowledge of how their program works in their community – find ways to tap into this knowledge during model development work.
- Adults are active learners, preferring to use reasoning and collaboration to find solutions. Actively engage stakeholders in model development – assign backwards mapping exercises or small group projects to identify program inputs, outputs, or outcomes.
- Adults want learning activities to be immediately applicable to our lives or work. Evaluators should demonstrate how theories and models can be applied to improve programs’ effectiveness.
Rad Resource: ActKnowledge’s Facilitator Source Book for Leading Theory of Change Development Sessions provides guidance on leading collaborative sessions to craft long-term outcomes and use “backwards mapping” to identify the earliest changes that need to occur.
Rad Resource: The Logic Model Workshop Toolkit, prepared by the REL Northeast & Islands, provides guidance for facilitating a logic model workshop; resources include a facilitator workbook, a participant workbook, and a slide deck.
The American Evaluation Association is celebrating Program Design TIG Week with our colleagues in the Program Design Topical Interest Group. The contributions all this week to aea365 come from our Program Design TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hi Patricia,
I enjoyed reading your post. I am a graduate student at Queens University and am currently taking a course on program inquiry and evaluation. We are in the process of creating our own program evaluation design so I am familiar with creating logic models and the theory of change. We have been discussing the variety of dilemmas that are present in evaluation, and one common dilemma that many of my colleagues have brought up has been around the role that program stakeholders have in the evaluation process. In your post, you mention that you include a large cross section of stakeholders when creating the logic model and theory of change, and this includes staff, beneficiaries, partners and funders. Do you find it difficult to come to an agreement at times with different people each having various interests in the outcome of this program? How do you account for individual biases?
Shulha and Cousins (1997) describe this issue when they state, “As we openly acknowledge, the swing toward collaborative models raises serious questions about the evaluator’s ability to maintain a sufficiently bias-free stance due to pressures emanating mainly from within the program community. While being undeniably important in any evaluation context, the question of understanding the complexities of, and the potential for, patterns of misutilization is especially pivotal in situations where evaluators work closely with program stakeholders.”
With my knowledge of collaboration up to this point, it is my understanding that in order to successfully collaborate there must be a common interest among participants. While all stakeholders have an interest in the same program, they may be affected in different ways by it. When creating your logic model with input from stakeholders, do you take the most common themes, and only include those? Or do you include all potential outputs and outcomes in order to create a program that impacts as many stakeholders as possible?
I would love to hear your thoughts!
Thank you,
Jenica
References:
Shulha, L., & Cousins, B. (1997). Evaluation use: Theory, research and practice since 1986. Evaluation Practice, 18, 195-208.
My apologies,
I should have addressed this to Sheila Robinson.
You were correct in addressing Patricia as the author. I’m the curator who posts most of the articles. Thank you!