Andy Rowe here. I work collaboratively to design sustainability-ready evaluations that recognize how human and natural systems are inextricably coupled. Most evaluations focus on the human system and ignore the natural system, thereby contributing to the looming extinction. But human system interventions have direct effects on the natural system; ignoring these direct effects causes evaluation to have a systematic and positive bias.
To illustrate the dominance of human system focus of evaluation, a scan of the two publications of the AEA from 1998 to 2016 yielded an average of one article a year involving the natural system in AJE and NDE combined. By contrast, and more in line with our growing awareness of climate and sustainability challenges, sustainability science publications generally showed a 15-20% annual growth from 1997-2007.
How did we get here?
The social sciences and knowledges undergirding evaluation are infused with the concept of DOMINION of humans over all other species, including dominion of European-origin peoples over other humans. This contributes to why we do not value what we take from the natural system, or cost what we return to the natural system through our production and consumption, part of why I argue that evaluation has a systematic and positive bias. By contrast, worldviews that regard all as connected and equal make more sustainable production and consumption decisions, notably Indigenous peoples. Thus, potentially, Indigenous evaluation can lead us towards a sustainability-ready evaluation function so long as evaluators do not colonize it with dominion-infused concepts and methods.
A simple illustration: plastic
It is almost certain that any intervention you evaluate consumes plastic. The MacArthur Foundation estimates that by 2015 the weight of plastic in the oceans will exceed the weight of fish. An estimated million plastic bottles are sold every minute worldwide. These and other sources of plastic are ingested by marine and terrestrial species which find their way into our food. Evaluation should provide guidance on reducing plastics, just as we do for the range of human system effects that we currently include.
Our responsibilities
Clearly human system interventions affect the natural system, and the resulting changes we have induced in the natural system affect us. This should matter to evaluators since the populations most vulnerable to climate change and adverse environmental effects are populations that are an important focus of the human system interventions we evaluate. Our future will include more extreme weather, fire and water inundation; the homeless, elderly and infants, those marginalized by race and gender will have greatest difficulty adapting to these ever more frequent and severe events.
Evaluators have both individual and professional responsibilities on climate and sustainability. We need sustainability-ready evaluation designs that consider how interventions affect both natural and human systems interactively.
Hot Tips:
- In theories of change map human and natural systems interconnections.
- Facilitate consideration of how to incorporate sustainability into the evaluation.
Sobering thought: Evaluation that is not sustainability-ready is, itself, not sustainable when we fail to address the major issues of our times.
Rad Resources:
- Read the summary of the most recent report of the Intergovernmental Panel on Climate Change (IPCC) synthesis for policymakers https://www.ipcc.ch/pdf/assessment-report/ar4/wg1/ar4-wg1-spm.pdf
- Just released – Fourth National Climate Assessment Volume II: Impacts, Risks, and Adaptation in the United States https://nca2018.globalchange.gov/
See how you can reduce your personal impact: https://www.nytimes.com/interactive/2019/04/30/dining/climate-change-food-eating-habits.html
The American Evaluation Association is celebrating Memorial Week in Evaluation. The contributions this week are about the concept of extinctions — those long past, recent, imminent, and foreshadowed. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Great insights Andy, thank you. I am sad that we were not able to discuss this topic together at CES 2019!
One of the challenges I find is that evaluators tend to conceptualize sustainability as the sustainability of program outcomes, rather than about how programs might impact environmental sustainability. It doesn’t help that the language of the DAC criteria (which many of us rely on) focuses on the former, so really changing the language of how we view and measure sustainability is also important for encouraging us to care about natural systems in evaluations. You alluded to this point but it might be helpful to explicitly talk about synergies and trade-offs between social and environmental goals; e.g. how a social intervention might indirectly help communities adapt to climate change.
I like your point Kylie about having a subsection for sustainability like we commonly do for equity (or perhaps better, integrating it throughout evaluation processes and products); actually, I think we can learn a lot from evaluation theory and practice related to social and gender equity. While we do not assess equity in a way that contributes to transformation (yet), we can learn from some of the challenges and opportunities in its evaluation and apply learnings to advance ‘sustainability-ready’ evaluation theory and application.
I am sorry everyone for not noticing the comments section to this post. Much time has passed and things are changing rapidly – but the extinction crisis remains of course. The pandemic has clearly shown the trade offs between economy and environment. Many development agencies have or are moving to mainstream climate, and encountering the tremendous shift that this requires, and the difficulty of reversing two centuries of extracting from the natural system.
Steven – the OECD DAC criteria provide general guidance for development evaluation and many of us strongly feel that sustainability of natural systems and dependence of the human on natural systems should have been recognised as a criteria. But OECD DAC are guidance have utility and do not prevent inclusion of sustainable development in evaluation, Still it would have been better if… And generally in evaluation the almost exclusive focus on the human system and topics therein colours everything – so sustainability means of programs for humans, and the climate crisis affects all interests so is perceived as a reason for ever more funding and priority – when in fact resources for human system interests will be strongly diminished if we are to avoid extinction and keep capitalism.
Kylie – the Canadian Evaluation Society has sustainability as one of three principles in the new strategic plan (along with equity and member services) and is assessing how to support the transformation of evaluation in Canada to be ready, able and interested to mainstream sustainability. I am working with Patricia Rogers, Jane Davidson and Dugan Fraser in an effort hosted by Better Evaluation and sponsored by CLEAR AA to develop guidance and materials to support evaluators in addressing sustainability in every evaluation – the effort is called Footprint Evaluation and we will be posting our first materials in the next few weeks on the Better Evaluation website.
Best to you all, thanks for your comments
Andy
In the post, it says “The MacArthur Foundation estimates that by 2015 the weight of plastic in the oceans will exceed the weight of fish.” Did you mean a different year? If not, does the weight of plastic exceed the weight of fish now (4 years later)?
Oops – you are right Rachel – it should have been 2050. MacArthur are applying the circular economy approaches of the International Resources Panel in advocating for solutions. The work of the IRP is well worth reading. Here is a source to the 2050 threshold:
https://www.theguardian.com/business/2016/jan/19/more-plastic-than-fish-in-the-sea-by-2050-warns-ellen-macarthur
Wouldn’t it be neat if every evaluation we did had a sub-section in the recommendations on “Sustainability Considerations”, whether these have been asked for or not. Same for “Equity Considerations”.
And I’ve said it before at CES2018, and I’ll say it again – maybe we need to adopt a 100-mile work diet where we stop hopping on planes everywhere. I’m guilty of this too but have been able to cut it down to almost no plane travel for work this past year. The train is kind of fun. A bit slower, but then no one said change was easy.