Hello. My name is Jindra Cekan, and I am the Founder and Catalyst of Valuing Voices at Cekan Consulting LLC. Our evaluation and advocacy network have been working on post-project (ex-post) evaluations since 2013.
Lessons Learned:
Most funders and implementers value interventions that have enduring impact beyond the project. We believe that the true measure of sustained impact and effectiveness can be measured only by returning after projects close. Our research indicates that despite more than $5trillion investment in international programming since 1945, fewer than 1% of projects have been evaluated for sustained impact.[1] After searching through thousands of documents online we found fewer than 900 post-project evaluations of any kind (including 370 publically that interviewed project stakeholders who are to sustain results once projects finish. Their views are key if we are going to meet the Sustainable Development Goals by 2030[2], for without such feedback our industry’s claim to do sustainable development falters.
This new type of evaluation is Sustained and Emerging Impacts Evaluation (SEIE). The focus is on long-term impacts plus both intended and unintended/ emerging impacts post closeout. This guidance comes from our global, growing database of post project evaluations, SEIE consulting and from a joint presentation at 2016’s AEA Conference, “Barking up a Better Tree”.
The guidance outlines:
- What is SEIE?
- Why do SEIE?
- When to do SEIE?
- Who should be engaged in the evaluation process?
- What definitions and methods can be used to do an SEIE?
Valuing Voices was just awarded a research grant from Michael Scriven’s Faster Forward Fund to do a desk study comparison of eight post-project (ex-post) evaluations and their final evaluations to better demonstrate the value added of SEIEs. The learning does not stop at post-project, as there are rich lessons for projects being currently funded, designed, implemented and evaluated.
Project cycle learning is incomplete without looking at sustained impact post-project, as sustainability lessons need to be fed into subsequent design. Opportunities abound from evaluating sustainability around the cycle:
- How is sustainability embedded in the funding, partnership agreements,
- What data is selected at baseline and retained post project and by whom,
- What feedback about prospects for sustainability is being monitored and how are feedback loops informing adaptive management, and
- When, how, with whom is project close-out and handover done.
Rad Resources:
The Better Evaluation site on SEIEs including examples from where impact was sustained, increased, decreased or new ones emerged;
Valuing Voices repository and blogs on post-project SEIE evaluations.
Great work on Exit Strategies that includes USAID/ FHI360/Tufts work on exit strategies, UK INTRAC’s resources on NGO exit strategies as well as a webinar on sustained impact, plus Tsikululu’s work on CSR exit strategies.
Underneath our work is a desire for accountability and transparency to both our clients (our donors and taxpayers) and those who take over (the national partners: governments, local NGOs, and of course the participants themselves).
[1] This is based on an extensive scan of documents posted on the Internet, as well as requests to numerous funders and implementing agencies through Valuing Voices’ networks.
[2] Currently EvalSDG is focused on building M&E capacity and amassing data on 230 indicators on indicators such as income, health, education etc but these are unrelated to the sustainability of development projects’ results.
The American Evaluation Association is celebrating International and Cross-Cultural (ICCE) TIG Week with our colleagues in the International and Cross-Cultural Topical Interest Group. The contributions all this week to aea365 come from our ICCE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.