Kirk Knestis (Hezel Associates CEO) here again with more thoughts about ways logic models can inform an evaluator’s work and potentially benefit the program managers and innovation developers with whom we work. One area where I think our contributions may often be underutilized is that of sustainability of the evaluand programs or innovations we study.
Lesson Learned – I confess that I’ve been guilty of neglecting “inputs” as they are typically illustrated in logic models. I’ve generally focused on defined activities and desired outcomes as priorities when making decisions about evaluation designs, data requirements, and analysis plans. Inputs were, in my view, simply part of the context within which an evaluand existed to be studied. Recent work of two of my colleagues, Andrew Hayman and Sarah Singer, has changed my perspective on this, particularly focusing my attention on inputs as the source of valuable insights into how a program might be sustained. Sustainability is often a concern for our clients, so this understanding can translate into additional value we offer as part of our evaluation services.
Rad Resource – Sarah and Andrew presented a Roundtable at Evaluation 2016, facilitating discussion about sustainability, how we can evaluate it, and how evaluation might help programs become sustainable. For this session (slides are available in the AEA Public Library), they defined “sustainability” as “continuing program activities or vital components of program activities to achieve intended outcomes without relying on future grant funding.” Their session identified obstacles to evaluating sustainability but more importantly, explored strategies to help a program be more sustainable, including by backward-mapping through its theory of action.
Hot Tip – As an example, consider this slice of a conventional style logic model:
Input (Resource) > Activity (Program Component 1) > Outcome A > Outcome B
If the evaluation finds Program Component 1 (an activity) to be mission-critical to immediate Outcome A, which is in turn required to achieve distal Outcome B, then that activity should arguably be sustained. If that activity requires a particular resource as an input, then sustainability requires (a) sustaining that input to support the crucial program component, (b) replacing that input with another providing similar support for the activity, or (c) modifying the activity so it can be delivered without (or with less of) the input. Regardless, planning for sustainability requires attention to inputs, and evaluators can help program managers or innovation developers plan ahead to structure the evaluation and data collection to that end.
Rad Resource – Other hints are shared in the notes from a similar conversation we facilitated at the 2015 conference of the Eastern Evaluation Research Society (EERS), examining distinctions between “evaluating sustainability” and “evaluating FOR sustainability.” Presented in terms of Challenges and Solutions, these ideas provide concrete ways evaluators might start to leverage logic models to meet another need for clients, developed from some of our recent projects.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.