Evaluation science and the rise of artificial intelligence by Robert Picciotto

Hello. I’m Robert Picciotto from the University of Auckland. I recently joined the organising committee of the International Evaluation Academy, a global movement of volunteers committed to evaluation policy influence, professionalism, and transformative impact across borders. This initiative is much needed at a time when science is under attack, public confidence in institutions is at all-time low, and public policy making is not rising to the challenges of climate change, rising inequality, and racial injustice faced by decision makers around the world. Addressing the ‘problems without passport’ of the post-Covid era will call for evaluation practice to grow, to transcend disciplinary boundaries, and to tap the exploding power of modern computers and Big Data.

The interconnected threats facing humanity do not obey the rationalist, predictable laws associated with the clock-world universe model that still dominates policy making. Systems thinking is needed and the same complexity paradigm that has invaded advanced mathematics and the natural sciences will have to be adopted by evaluators in pursuit of sustainable solutions to the intricate problems faced by decision makers in a world where capital knows no borders, and the problems of others have become our own. Simple linear models of causality are no longer appropriate. The time has come for a ‘complexity turn’ to take place in evaluation (https://journals.sfu.ca/jmde/index.php/jmde_1/article/view/643)

A novel way to explore causality has emerged.  New mathematical notations are now available to eliminate the ambiguity of conventional equations. The feedback loops and the dots and arrows of theories of change graphs (even when they resemble spaghetti bowls) can now be translated into equations that ‘mathematize’ presumed causal linkages and facilitate computer simulation of alternative conjectures, however elaborate. This opens the way for systematic experimentation that tests the validity of alternative theories of change through a combination of rigorous logic and computer-friendly semantics. As a result, evaluation can now move beyond the theory-free approach of statisticians and randomized control trials. This is nothing short of a revolution in evaluation practice.   https://journals.sfu.ca/jmde/index.php/jmde_1/article/view/507.

What then lies ahead in the evaluation world? As in other sectors of modern society, artificial intelligence (AI) technologies are exploring new frontiers of knowledge and helping to perform a wide variety of tasks in far more reliable, efficient, and effective ways than were possible without them.  The science of intelligent machines has already made major contributions to agriculture, industry, construction, and trade. Robots perform dangerous, repetitive work efficiently and safely. They excavate the earth, harvest crops, assemble cars, shelve goods, take inventory, deliver food, disinfect hospital rooms, etc. In partnerships with human agents, they have also begun to drive cars, produce art, help autistic children, provide company to the elderly, and they even perform religious ceremonies.

AI researchers have also begun to create algorithms designed to take over vast amounts of work in white collar fields. Knowledge work that involves pattern recognition, large amounts of data and decision making are particularly vulnerable to AI capture. This will not replace human beings at the high end of the evaluation profession: robots do not have common sense, intuition, or ethics but they can assist expert evaluators in their work. Thus, the evaluators who are familiar with AI concepts and tools are likely to replace those who do not: artificial evaluation will become an essential component of high quality, expert evaluation. Hence, familiarity with data science and AI will gradually emerge as basic evaluation competencies.  

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.