The posts for this week come from the Digital Data & Technology AEA Conference Working Group, and share how digital data and technology are factoring into reshaping evaluation.
Hi all, I’m Linda Raftree and I have been leading the MERL Tech community of practice since 2014. MERL Tech is a term we use to describe technology-enabled Monitoring, Evaluation, Research, and Learning (MERL). Our vision is a world where the use of digital tools, technologies, and data for MERL in humanitarian, development, human rights, peacebuilding, and social impact programming is inclusive, ethical, responsible, safe, and sustainable.
As digital approaches for MERL become the norm, we believe it is important that they support equity, equality, and justice, and that they avoid doing harm or contributing to greater levels of surveillance of historically marginalized communities, activists, and civil society organizations. We work to strengthen organizational and sector-level skills, knowledge, and capacities in the areas of digitally enabled MERL; improve MERL of digitally enabled programming; and advocate for ethical innovation and responsible use and management of technology and digital data.
In 2019, we looked back at the evolution of 5 years of digital approaches to MERL and found that we could organize rapid sector changes into 3 waves:
Lessons Learned
Then, in 2020, COVID happened, and digital tools became even more critical for MERL, because suddenly everything went remote. As MERL Tech became an even more established norm for evaluators, we highlighted at least three key areas to watch:
- Access and inclusion. It is estimated that 67% of the global population can now access the Internet (at least intermittently). But the most vulnerable are often still left out. This skews data collection towards the elite and the most ‘convenient’ methods, affecting data quality. Big data also has its challenges, considering that historical data sets used for evaluation or for ‘predictive analytics’ risk encoding bias and injustice into their decision-making.
- Data protection. The digitization of MERL means that we collect data in new ways, this often helps create efficiency and improved data management, analysis, and use. Digital data brings different kinds of risks than paper data, however, and the evaluation community is not always up to speed on how to ensure data privacy and protection. The sector has often focused on getting data more than on protecting it. This is an important area that needs more attention.
- Data quality. While in some ways, digital data can help with data quality, remote data collection raises several concerns. When there is less face-to-face contact, there may be less trust among respondents leading to lower quality responses. It is also difficult to validate the data and findings given that there are fewer contextual cues during an interview, for example, and there is little ‘ground truthing’ of findings. Additionally, in low-income communities, phone sharing (meaning you might not get the same person each time you reach out) and loss of contact with respondents can also be a challenge.
The MERL Tech Community has learned a lot about digitization and remote MERL since 2020, including about where digital adds value and where it fails to go far enough in its support. We still believe that regardless of how much or how little tech is involved, people and planet-centered approaches are the real gold standard.
Get Involved
- Check out the MERL Tech website for resources and information about digital MERL
- Sign up for the ICT-enabled M&E strand at the South Africa M&E Association (SAMEA) conference in September
- Join us for conversations on digital data, new data approaches, and data responsibility at the South Asian Community of Evaluators conference in October
- Sign up for AEA’s Integrating Technology into Evaluation (ITE) TIG for discussions of digital approaches to MERL! We’ll be sharing at two Eval22 sessions as well: African Perspective on Responsible Data Governance in M&E and Trends in African MERL Tech: Insights from a Landscape Scan.
The American Evaluation Association is hosting Digital Data & Technology Week with our colleagues in AEA’s Digital Data & Technology Working Group. The contributions all this week to AEA365 come from working group members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.