Hello, AEA365 community, and happy Conference Week from Indianapolis! The AEA staff have been working overtime to prepare for our biggest event of the year. Whether you will be joining us for the conference or not, you can keep up with our happenings via the AEA365 blog. See you around!
-Liz DiLuzio, Lead Curator
Hi, I’m Kim Norris, Monitoring, Evaluation and Learning (MEL) Director for American Institutes for Research (AIR)’s International Development Division. Part of my role is to lead a MEL practice. As part of our initial strategy, our practice team determined to focus on localizing our work. For us this means we seek out ways to increase local partnering and leadership in and around MEL efforts – from business development to MEL direction and execution. This involves local team leadership, capacity strengthening and engagement on local terms.
Our team is keenly aware that stories are best told by those who have lived them, and that we are at risk of losing the evaluation story without significant local engagement. We have learned how using the Most Significant Change method (MSC) can more actively involve local participants in identifying, analyzing and interpreting significant changes since program inception, and can help to uncover hidden and emergent aspects of an intervention’s relevance and effectiveness.
MSC, a participatory evaluation approach, focuses on collecting stories of significant change from individuals or groups who have been impacted by a program or project. Stories are analyzed and used to identify the most significant changes of an intervention from the perspectives of participants. In addition to giving voice to participants, MSC enables them to shape subsequent evaluation processes through their initial responses.
MSC is particularly beneficial in settings where desired changes are complex, diverse, and difficult to measure using traditional quantitative methods, and in situations where we need to capture subjective experiences and perspectives of beneficiaries. As a qualitative method, MSC provides insights into why as well as what has changed significantly. MSC can be used to supplement, check results, and expand on learning from more traditional methods, thereby enriching the local story.
Lessons Learned
Adapting Most Significant Change to Enrich the Local Story: A key strength of MSC is its malleability – it can be tailored to fit different contexts and accommodate specific evaluation needs.
Strategies for adapting MSC to different situations are based on the three key elements of the method: 1) collecting stories, 2) systematically selecting stories of change, and 3) a feedback loop that continues throughout the process, whereby story selectors discuss and select which of the reported lived experiences is most significant, and why. The ‘why’ question challenges participants and story selectors to discuss values behind the changes reported, and to formulate thoughts and engage in deeper discussions about why they believe a change to be most significant. Focusing MSC on enriching the local story:
- participant selection can ensure inclusiveness and equitable inputs;
- story collection methods can be customized to suit the cultural and social norms of the community to respect local traditions and values and to be inclusive of marginal groups;
- analysis and interpretation of stories can be aligned to local context by, for example, applying culturally appropriate frameworks or involving local experts in the process; and
- feedback and dissemination of findings can be made accessible and meaningful to the local community by using local languages, and community events to share results.
Rad Resources
The Davis, R., and J. Dart publication, The ‘Most Significant Change’ (MSC) Technique: A Guide to Its Use (2005), offers a description of the technique and an initial guide to its use.
Published in Global Health: Science and Practice (2022), Lessons Learned From the Use of the Most Significant Change Technique for Adaptive Management of Complex Health Interventions, showed that using the MSC technique led to voices being heard across all levels of the health system, increased learning and collaboration across project stakeholders within and across countries, and led to project leaders taking concrete adaptive management actions.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.