Hello, my name is Ayesha Boyce. I am an associate professor within the Division of Educational Leadership and Innovation at Arizona State University.
My favorite evaluation lesson to facilitate is logic modeling. Developing an in-depth understanding of the context and culture of the evaluand is paramount for a successful evaluation. Further, successfully mastering logic modeling is useful for evaluation and program planning alike. In addition to helping stakeholders understand formative and summative outcomes associated with their program, evaluators can facilitate discussions about the logic, theory, and purpose of their program. All evaluators should aim to play an educative role as part of their practice. Overall, I am a big fan of everyone knowing how to develop logic models and I enjoy working with stakeholders and students alike to surface program theory and make implicit values explicit.
Hot Tip:
Perfect practice makes perfect. Learning how to logic model takes time! I have students develop their skills by workshopping logic modeling multiple times in class and having a logic model homework assignment. We begin by reviewing the meaning of inputs/resources, activities, outputs, outcomes, and impact.
Hot Tip:
Real world and program contexts are key. We then walk through the popular example of visiting family during the holidays presented in the W. K. Kellogg Foundation Logic Model Guide. Next, I open an empty logic model table and present a scenario of purchasing and moving into a house. Beginning with inputs, I ask students to tell me what resources are needed for purchase. We brainstorm to fill the cells together. Some examples students provide include needing a realtor, good credit, and an idea of where you want to live. We then work through the activities, outputs, outcomes (short and mid-range), and impact. After we have developed a logic model as a group, I then show them an example logic model I have completed for that same context.
After we complete the buying/moving into a home logic model exercise, we repeat the progress with an example of a real program.
Hot Tip:
Together alone and alone together. After we have done the exercise twice as a whole class, I have students break into small groups and complete another logic model for a second program. Then I have them complete a third logic model for a different program as homework.
Lessons Learned:
No vacuums! Neither programs nor their evaluations operate in a vacuum. We must consider context when developing logic models, including community, historical, political, environmental, and organizational cultures and factors.
Lots of reviewing to do! I remind students that when they are developing a logic model they should review written program proposals, guidelines from funding agencies, program materials, management materials, and past evaluation reports.
Outputs are off-putting! The column that usually proves the most challenging for newer evaluators is “outputs.” Students have difficulty differentiating between outputs and outcomes, so I spend extra time overviewing what outputs are and having students practice coming up with outputs for each context.
Keep it going! I utilize the same three contexts from the logic model exercise throughout class. So, when it is time to develop an evaluation framework, data collection matrix, and timeline students can workshop these evaluation components while referring back to the logic models from these activities.
Rad Resources:
CDC Identifying the Components of a Logic Model
Frechtling, J. A. (2007). Logic modeling methods in program evaluation (Vol. 5). John Wiley & Sons.
Suber, L. (2019). What are logic models, and when should you use them? Retrieved from https://www.ies.ncsu.edu/blog/what-are-logic-models-and-when-should-you-use-them/
W. K. Kellogg Foundation Logic Model Guide
The American Evaluation Association is hosting Teaching of Evaluation TIG week. All posts this week are contributed by members of the ToE Topical Interest Group. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.