Hello, AEA365 community! Liz DiLuzio here, Lead Curator of the blog. This post is a continuation from Transformational Eval Week (12/10/23-12/16/23), a topic so hearty that we couldn’t contain the posts to just seven. Please enjoy.
Hi. My name is Romeo Santos, a council member at the International Evaluation Academy. I’m one of the founders of the Asia Pacific Evaluation Association, and I served as president in 2018-2019. I started dabbling in monitoring and evaluation (M&E) in 2000.
I wrote this blog as a form of reflection. You may agree or disagree with the points I raised. However, I’m open to critiques and suggestions. Please feel free to contact me.
The present evaluation environment: need for transformation
The following are mere observations. By the way, this post does not intend to put down, besmear, nor denigrate any person, entity, or idea, but to learn from them. It only underscores a real need for transformation. Sometimes, it pays to laugh at our own skeletons as they pop out of our closets.
- There is this tendency to put evaluation on top of a pedestal, kind of a cult-like adoration. Often, evaluation is portrayed as if it were the only source of knowledge. In reports, it is unintentionally depicted as one that gives the most benefits and not the project implementation outcomes.
Some pundits noted a growing number of disenchantment and discontent in the current evaluation ecosystem. Sometime ago, a writer on an online platform strongly demanded for an “evaluation-free zone”! Another one quipped about “RBME madness”! Whatever the context, we feel that there’s some kind of problem here, something’s stealthily suffocating!
- At times, evaluation reports are made no different from academic writing. Some reports appear wonderful in form but short in substance. There are those that look like pile-it-up works, covering almost everything with little regard to scope as per the terms of reference (TOR). TORs themselves display ‘half-baked’ features, likely written by amateurs. In fact, it’s sad that many evaluation commissioners-managers do not have actual experience doing evaluation. While many sectors focus on creating demand for evaluation, the quality of the supply side appears wanting.
- There is this penchant for coining terms, perhaps to innovate or attempt at originality. Terms like sustainability, leverage, mainstream, programmatic, etc., have become buzz words that populate the evaluation vocabulary. And yet, we usually turn heads upon hearing the pun word ‘UN-speak’!
- The M&E is an open field for migrants. One evaluation guru noted that “anyone can pose as an evaluator”. In fact, it is not uncommon to see former state officials acting as expert evaluators, or RBME experts today. Whereas, during incumbency, a few of them were antagonists, or in some way hindered the adoption of evaluation. This fact implies that professionalization is a significant part of the needed transformation.
The dilemma: which one to transform, evaluation or society?
Some thinkers may opine that socio-economic transformation, coupled with transformative change in ‘thinking about self and the world’ is needed to create a healthy environment for evaluation. In contrast, transformational evaluation was defined as one that pursues ‘the goal of bringing society to a point of greater equity and justice’. These 2 ideas, being valid, put us in a quandary, which this post cannot fully elaborate on or resolve at the moment. But one question stands out: Which one should come first, transformation of society, or transformation of evaluation?
The depth of change: can evaluation sink rock bottom to effect real transformation?
This query might pose a big dilemma again. Some experts believe that “…we still lack a broad understanding of and agreement on what constitutes transformational change…xxx… how it can be termed successful and effective in …xxx… .” Obviously, this inference makes sense, but then, how do we identify effective transformation when we see one?
Takeaway
Indeed, the task of bringing about transformational change in society is daunting, especially if we see evaluation as a tool to accomplish it. What I wrote here are just reflections, partly expressed thru the cited observations. I intend to expand more but space is limited. Nonetheless, whenever I write about transforming evaluation, I will always watch out.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.