Hello! It’s Linda Raftree and Michael Harnar here, both fresh from the MERL Tech Conference in Washington, DC September 4-6, 2019.
Linda is the organizer of the conference, which brings together a multidisciplinary community of evaluators, non-profit professionals, data scientists, government agencies and funders to learn and share their work and their challenges in responsibly using new technologies for MERL (monitoring, evaluation, research, & learning).
Michael is an Assistant Professor of Interdisciplinary PhD in Evaluation at Western Michigan University, where the focus is on developing the next generation of evaluation scholars and practitioners.
The two of us have joined forces with several others on a Research on Evaluation initiative that is looking at the field of MERL Tech and building on Linda and Michael Bamberger’s 2014 paper, Emerging Approaches: Monitoring and Evaluation in a Tech-Enabled World. Spoiler: so much has changed in the past 5 years!
In 2014, the buzz was all about SMS, crowdsourcing, citizen journalism, real-time data, feedback loops, and mobile data gathering. People were experimenting with all sorts of new approaches, but because most things were experimental, there was not much of an evidence base outside of blog posts and grey literature. In 2019, these approaches continue to be used, but we’re also hearing about artificial intelligence, machine learning, text and sentiment analysis, the “dark web”, remote sensing, drones and the blockchain. After 5 years of running the MERL Tech Conference, we were fielding questions about the state of the field, but our responses were not supported by systematic research.
So, we gathered an all-volunteer team to explore the evidence base on three fundamental questions (where are we, where have we been, and what have we learned?) across three broad areas:
- Rapid scoping review of the academic literature on MERL Tech: We assume using tech in MERL can allow for better, faster, cheaper, more scalable MERL and higher quality data but what can we reliably say about the use of technology for these approaches?
- Big Data: We hear of big data as the next frontier, but where are the real examples? Are Evaluators actually using Big Data for MERL? Why or why not? And should they be?
- Emerging Approaches in MERL (such as AI, ML, drones, blockchain, sentiment and text analytics). What are the emerging technologies and approaches to MERL that we should be looking at? What do we know about them? Who is actually using them and why?
Hot Tip: Join us at our AEA Presidential Strand Panel to learn more
Our teams presented preliminary findings at MERL Tech DC and will have more to share at AEA in Minneapolis. Come join the conversation and engage in defining how we want technology and innovation to look in the MERL space in the future. We’ll share findings at our presidential strand panel (#2519) “What’s happening with Tech in Evaluation? A state-of-the-field baseline to launch discussions about the future of evaluation” Saturday 11/16 10:15–11:00 a.m.
The American Evaluation Association is celebrating Integrating Technology into Evaluation TIG Week with our colleagues in the Integrating Technology into Evaluation Topical Interest Group. The contributions all this week to aea365 come from ITE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.