This week’s posts highlight reflections from the Global Evaluation Initiative (GEI), a global network of organizations and experts working together to support the strengthening of monitoring, evaluation, and the use of evidence in developing countries. GEI uses an integrated systems-based approach and works closely with governments, evaluation professionals, and other stakeholders on efforts that are country-owned and aligned with local needs and perspectives.
-Liz DiLuzio, Lead Curator, AEA365
Hi, I am Heather Bryant, a member of the Global Evaluation Initiative (GEI) Global Team. The GEI supports developing countries in strengthening their monitoring and evaluation (M&E) systems to help governments gather and use evidence that improves the lives of their citizens. The GEI believes that to effectively support countries in this process and to be able to provide tailored, context-specific advisory services, it is first necessary to understand the existing systems that affect M&E in the country. To assist in this effort, the GEI Global Team, in collaboration with the global network of Centers for Learning on Evaluation and Results (CLEAR), developed the Monitoring and Evaluation Systems Analysis (MESA) diagnostic tool. The MESA is a tool that guides stakeholders (e.g., government entities, evaluation professionals, civil society) in gathering, structuring, and analyzing information on the current capacity of their country’s M&E ecosystem. Since the tool’s launch in early 2022, the GEI Global Team and colleagues in the CLEAR network have been using it to help identify what is working well, what needs to be improved, and to inform capacity-development strategies meant to strengthen the systems that enable M&E to flourish. We have learned a few things along the way – both in the development and in the use of the tool.
Lessons Learned
Bring multiple voices to the table. When developing the MESA tool, we started with a small, but diverse team, and looked at existing tools from different contexts for design inspiration. We also benefited from inputs and perspectivesfrom people working in a variety of countries,which helped us design a tool that can be adapted to local needs and situations. When conducting a MESA, we have also found it essential to include multiple voices and views from diverse stakeholders throughout the process. This helps build relationships, nurtures collaboration, and provides a more comprehensive picture of the country’s M&E ecosystem, which leads to a more effective and sustainable strengthening strategy.
Engage and prioritize local expertise. The MESA is not intended to be an assessment delivered by an external expert, but rather a co-creation with all participating stakeholders. While international experts may lead a MESA process, especially in contexts where there is not yet a strong culture of evaluation, it is critical to include local expertise on the team. A MESA should build an understanding of what already exists in the country, what has already been tried and done, where the differences between de jure and de facto practices lie, and where there are opportunities and entry points for change – all of which may not be immediately apparent to an external expert. Local team members can also help ensure that the MESA itself is an advocacy and capacity development intervention by translating unfamiliar concepts – such as evaluation – into appropriate vocabulary during interactions with country stakeholders. Local experts can also help to make sure that the MESA process is conducted using culturally appropriate methods. The MESA process in the Solomon Islands benefited from such local expertise. For example, the team used tok stori sessions – facilitated yet fluid discussions that involve narrated stories about experiences and observations – in the data collection phase; iola – an indigenous approach to organizing research along the divisions of a canoe – in data analysis; and, rara’aba or a “calming of the nerves” meeting as part of the stakeholder validation process.
Tell a good story. The MESA report, which is a document that summarizes the findings of the MESA process and proposes next steps, must tell a good story. The MESA tool points to areas of inquiry and suggests guiding questions, but it is not enough to simply gather data and assemble it under the proposed headings. One has to “harvest” the data to craft a compelling story that will build stakeholders’ interest in enhancing and strengthening their M&E system. Each MESA team must adapt the story to the local culture and bring their own creativity to the process and the final product. A MESA report that is easily understood and resonates with decision-makers is more likely to make an impact and inspire change.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.