Hello. My name is Bob Picciotto and I am on the Executive Committee of the International Evaluation Academy. As Director General of the World Bank’s Independent Evaluation Group (IEG) for two consecutive terms (1992-2002), I was fortunate to report to the Board of Directors. In parallel, management carried out its own self-evaluations under the oversight of a Vice President. The same governance model is still in place, and it allows IEG evaluators to tell it like it is. This practice is modelled after the relationship between corporate accounting and auditing.
Such freedom is exceedingly rare in contemporary evaluation practice. In the market society, most evaluators are fee dependent. Evaluation today is user-directed. The ‘independent consultant’ designation is an oxymoron. Evaluation is a commodity and evaluators lack professional status, i.e., they do not enjoy full control over their own work. Evaluation commissioners set the rules of the game, and evaluators must market themselves as consultants. To make a living, they must be responsive to their paymasters.
This governance model is inadequate given the unprecedented challenges currently facing the world: evaluation today must be transformative to address the ‘problems without passport’ caused by extreme inequality, global pandemics, and climate change. As cogently articulated by Steve Waddell, et.al[1]., fundamental transformation requires a combination of destruction and creation on the one hand and confrontation and collaboration on the other.
| Destruction | Creation | |
| Collaboration | Missionaries (Unilever) | Partners (Gates Foundation) |
| Confrontation | Warriors (Greenpeace) | Entrepreneurs (Ashoka) |
In other words, the predicaments of our era call for a mix of confrontation and collaboration. Complexity theory and systems thinking help select the right evaluation governance model. They evoke interaction between order parameters located inside the evaluandand control parameters located outside it. Positive feedback generates instability and challenges hierarchy, while negative feedback promotes stability and strengthens hierarchy.
It goes without saying that new ways of funding evaluation will be required to allow evaluator-directed evaluation to take place without interference from decision makers. Only then, will evaluation break free from the chains of fee dependence that are constraining evaluation from speaking truth to power. To be sure, user-directed evaluation has value where the authorizing environment favours change and/or where only minor course corrections are called for. This is the northeast quadrant of developmental evaluation where interdependence reigns and evaluators can partner with decision makers.
But user-directed evaluation is inadequate when fundamental reforms are called for. The predicaments of our era require collaboration, but they also call for confrontation. This is when evaluator- directed evaluation comes into its own. What if vested interests have captured the enabling environment while the organization within which the evaluator is embedded is dedicated to social transformation? In such circumstances, user-directed evaluation is adequate and calls for alliances with progressive civil organizations focused on advocacy.
Where on the other hand the enabling environment is democratic, while the evaluand has been captured by vested interests, ethical evaluators should engage in adversary evaluation which implies resort to an evaluator-directed model. Equally, when the enabling environment is undemocratic and the decision-making organization is controlled by vested interests, only evaluator- directed evaluation is appropriate – a subversive evaluation approach implemented in league with progressive local community organizations and civil society organizations.
| Destruction | Creation | |
| Collaboration: user-directed evaluation | Advocacy Evaluation | Developmental Evaluation |
| Confrontation: evaluator-directed evaluation | Subversive Evaluation | Adversary Evaluation |
In sum, evaluation governance should strike the right balance between user-directed and evaluator-directed evaluation taking account of the characteristics of the strategic context and of the authorizing environment. Sticking with the current model of evaluation as a market good in all circumstances is simply not good enough. Evaluation should at long last be governed to serve citizens rather than privileging managers. To become a public good, evaluation should be liberated.
[1]https://www.researchgate.net/publication/279803248_Large_System_Change_An_Emerging_Field_of_Transformation_and_Transitions.systemic
The American Evaluation Association is hosting International Evaluation Academy (IEA) Week. The contributions to AEA365 this week are all related to this theme. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.