Welcome to sustainability! I’m Jindra Cekan/ova, 30+ years in global development, founder of Valuing Voices, Sustainable Solutions for Excellent Impact.
There is a compelling need to prove international projects and programs are as sustainable and long-term as we promise. However, evaluators of foreign aid during project implementation have a very limited set of tools with which to assess projected and achieved sustainability (defined by OECD as “the extent to which the net benefits of the intervention continue, or are likely to continue”) amidst the project’s lifecycle. Most claims rest on estimates of projected longevity of results and anticipated trajectories of ‘sustainability’s benefits’ resting on conjecture and hopes. However, these OECD measure for estimating the likelihood of sustainability have not been tested for reliability or validity.
In order to better evaluate sustainability, organizations would benefit from clarity around definitions and time:
- What do we mean by ‘sustainable’ and for how long?
- Defined by whom?
- How can we make the likelihood of sustainability better, pre-exit?
Different stakeholders use the term “sustainability” in different ways, and the concept of longer-term sustained impacts from interventions is frequently blurred by capturing project and program outputs only at project closure and projecting these will remain… sustainably developed. Information about actual longitudinal sustainability is limited as well: fewer than 1% of all international development projects funded by bilateral donors and international NGOs have been evaluated years after closure (Cekan J. 2015, “When funders move on”. Stanford Social Innovation Review.). Ex-post assessments of project and program sustainability are so infrequent that evaluators face a lack of information on post-project trajectories. This is with the exception of the Japanese Development Agency JICA who is a powerhouse of over 2500 ex-posts (some 3 years post-closure and some returning 4 years later again), but with no public meta-evaluation available across this hoard of learning, and a handful of USAID ex-posts in water/ hygiene, few dare to return. Even fewer ask what emerged from local efforts to sustain after resources, partners end.
Those that returned have done so between 2-30 years ex-post. Those 39 organizations that asked participants and partners for their evaluative voice about what was sustained, why/not and what emerged are featured as catalysts. While there are broad lessons on what to do better featured in Valuing Voices’ “Building the evidence base for post project evaluation report” to Michael Scriven’s foundation, including starting to map methods clear evaluability assessments, there needs to be enough of a dataset of ex-posts to compare results.
Rolling back expectations and claims via evaluation is important regarding climate change sustainability claims made by the Global Environment Facility Trust Fund (see Rad Resource). One of my favorite evaluators, Ian Davies challenges our development industry and us evaluators that “understanding and practicing evaluation [is] an activist process for sustainability, freedom and justice.“
We must exit more sustainably. More can be done to plan for the exit from the onset with local partners. See Valuing Voices’ checklists and valuable guidance by Stopping as Success (CDA et al) and evaluating readiness for exit in the hot tips. Start now, keep asking!
Hot Tips
- OECD Detailed explanation of Sustainability
- Checklists to help donors, implementers Exit for Sustainability
- Tips from USAID-funded Locally-led Transition series
Rad Resources
For a deeper understanding of ex-post sustainability and measurement:
- Sridharan and Nakaima’s Incorporating Planning of Sustainability into Theories of Change
- Cekan and Legro’s Verifying the Sustainability of Climate Change Mitigation Results
- Ian Davies’ Evaluation after COP26: Reflections, criticisms and opportunities
- JICA ex-post evaluations on loans and technical assistance grants, and
- USAID Global Waters six ex-post evaluations.
The American Evaluation Association is hosting International and Cross-Cultural (ICCE) TIG Week with our colleagues in the International and Cross-Cultural Topical Interest Group. The contributions all this week to AEA365 come from our ICCE TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.