APC TIG Week: Carlisle Levine on Using the Most Significant Change Approach to Explore Potential Evidence of Impact in Advocacy and Policy Change

Hello! I’m Carlisle Levine, President and CEO of BLE Solutions, LLC. We offer evaluation, applied research and technology services to help organizations increase their effectiveness and contribute to better outcomes. I specialize in global advocacy, peacebuilding and strategic evaluation.

A tremendous challenge in advocacy evaluation is identifying links between advocacy activities and changes in people’s lives, given the many factors that are involved and the time it takes for change to come about. The Most Significant Change approach can help respond to this challenge.

Rad Resource: The Most Significant Change (MSC) approach, an inductive, participatory outcome monitoring and evaluation approach, was developed by Rick Davies and then widely publicized in a guide co-authored with Jess Dart. It uses storytelling to gather evidence of intended and unintended, as well as positive and negative change. The stories are then reviewed and analyzed by a core team to identify the most significant change from their point of view. Importantly, MSC is not a standalone method. Rather, it can point to outcomes that require further validation using more deductive methods.

The approach involves 10 steps, according to the MSC Guide:

MSCStepsGraphic.Levine

Lessons Learned

  • In evaluating advocacy efforts, I first use methods that help me identify the contribution that advocacy efforts have made to policy changes. I then use MSC to explore early evidence of how those policy changes are affecting people’s lives.
  • In my design, I do not define domains of change, but wait to see what domains emerge from the stories themselves.
  • By triangulating a storyteller’s story with information provided by people familiar with the storyteller’s life, I increase the story’s credibility.
  • With my clients, I use the selection process to help them understand the variety of changes in people’s lives resulting, at least in part, from their targeted policy change. I also conduct a meta-analysis that shows them trends in those changes. With this information in hand, they can reinforce or adjust their policy goals and advocacy efforts in order to contribute to the types of change they most desire.

Hot Tip: To build trust with storytellers, I partner with story collectors who speak their language and are familiar with their context. The more storytellers believe a story collector can relate to their reality and will not judge them for it, the more open storytellers will be.

The American Evaluation Association is celebrating APC TIG Week with our colleagues in the Advocacy and Policy Change Topical Interest Group. The contributions all this week to aea365 come from our AP TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

1 thought on “APC TIG Week: Carlisle Levine on Using the Most Significant Change Approach to Explore Potential Evidence of Impact in Advocacy and Policy Change”

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.