Hi all! I’m Juha Uitto, Deputy Director of the Independent Evaluation Office of the United Nations Development Programme (UNDP). I’ve spent many years evaluating environment and development in international organizations, like UNDP and the Global Environment Facility (GEF).
As we all know, evaluating sustainability is not easy or simple. Sustainability as a concept and construct is complex. It is by definition multidimensional encompassing environmental, social, cultural, political and economic dimensions. It cannot be evaluated from a single point of view or as just one dimension of a programme. Apart from the above considerations, sustainability refers to whether the programme or intervention that is the evaluand is in itself sustainable. Sustainability evaluation, must take all of the above into account.
At its simplest, sustainability evaluation would look into whether the intervention would ‘do no harm’ when it comes to the various environmental, social, cultural and other dimensions that may or may not be the main target of the programme. At this level, the evaluation does little more than ensuring that safeguards are in place. The evaluation also has to look at whether the intervention itself was sustainable, i.e. whether it has developed exit strategies so benefits will continue beyond the life of the intervention.
But this is not enough. It is essential for evaluations and evaluators to be concerned with whether the evaluand makes a positive difference and whether it has unintended consequences. In environment and development evaluation a micro-macro paradox is recognized: evaluations show that many individual projects are performing well and achieving their stated goals; yet the overall trends are downward. There are lots of projects focused on protected areas and biodiversity conservation; still, we are facing one of the most severe species extinction crises ever. Many projects successfully address climate change mitigation in various sectors ranging from industry to transportation to energy; still, the global greenhouse gas emissions continue their rising trend. It is not enough for evaluators to focus on ascertaining that processes, activities, outputs and immediate outcomes are achieved.
Lessons learned: In evaluating environment and poverty linkages, one should never underestimate the silo effect. Sustainable development requires a holistic perspective but few organizations operate that way. People have their own responsibilities, priority areas, disciplinary perspectives, partners, networks, and accountabilities that often preclude taking a holistic perspective. Evaluators must rise above such divisions. An evaluation – such as the Evaluation of UNDP Contributions to Environmental Management for Poverty Reduction – can make a major contribution to how an organization acknowledges, encourages and rewards intersectoral and transdisciplinary cooperation.
Rad resource: All UNDP evaluation reports and management responses to them are available on a publicly accessible website, the Evaluation Resources Centre, and independent evaluations at Independent Evaluation Office of UNDP.
The American Evaluation Association is celebrating Environmental Program Evaluation TIG Week with our colleagues in the Environmental Program Evaluation Topical Interest Group. The contributions all this week to aea365 come from our EPE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
As a current undergraduate student still exploring opportunities in the psychology field, your articulation of the complexities of sustainability evaluation were eye opening. As one interested in the environment, I deliberate how to positively influence the environment we, as humans, inhabit. As an organism, humans are unique in the ability to continue to create innovative ways to deplete natural resources contributing to the imbalances we all see and feel. I am left with the notion that the best we can work toward is reducing our impacts to forestall the inevitable population impacts. The planet will survive us. So is sustainability’s role to allocate resources as equitably as possible for as long as possible?
Thank you, I absolutely agree that “It is essential for evaluations and evaluators to be concerned with whether… [it] makes a positive difference and whether it has unintended consequences.” So rarely do we go ask about these after projects close… what has UNDP learned about these positive differences and unintended consequences?
Also does UNDP go back after close-out 2, 5, 10 years later to see the self-sustainability of project activities beyond just exit strategies? What is the true life of the project in the view of the NGO/ community that ‘took over’ (often without resources to sustain activities)? What has UNDP learned from going back and how does it apply these lessons?
Thanks, Jindra
The elusive quest for learning… UNDP like other organizations I am familiar with finds it difficult to learn from evaluations. It’s perhaps even harder with a highly decentralized organization with some 139 quite autonomous offices around the world. There are mechanisms put in place to facilitate learning – mandatory management responses to evaluations and a follow-up system to them being perhaps the most important.
There is no systematic practice of ex-post evaluations. In our country-level evaluations (ADRs) we take a longer view, looking usually at a period of about eight years and trying to establish what transpired. The global thematic evaluations likewise attempt to track the results of interventions in a particular area over a longer period of time. Incidentally, we’re just conducting a meta-synthesis of all the ADRs (close to 80 of them) carried out over the past decade.
Thank you, thank you, thank you! Your discussion of how programs (in silos) can achieve programmatic objectives and yet not turn the curve on long term population goals is illuminating. Our organization has adopted Mark Friedman’s Results Based Accountability approach where programmatic work is intended to not just achieve short-term and intermediate-term program objectives but also to contribute to “turning the curve” on long-range population indicators. “Contribute” is the key word rather than “cause.” Until we join together to focus our efforts on contributions to these long-range measures, we will continue to fall short of reaching our key goals and all will suffer. Until positive change that benefits all outweighs short-term personal gains (attractive as these might seem at the moment), we will continue to grow our global challenge rather than diminishing them.
Thanks very much for your kind comment. Fully agreed.
Juha