APC TIG Week: Carlisle Levine on Using Contribution Analysis to Explore Causal Relationships in Advocacy and Policy Change

Hello! I’m Carlisle Levine, an independent evaluator specializing in advocacy, peacebuilding and strategic evaluation. I led CARE USA’s advocacy evaluation and co-led Catholic Relief Services’ program evaluation.

A big challenge in advocacy evaluation, because of the many factors that influence policy change and the time it takes for change to come about, is drawing causal relationships between advocacy activities and policy outcomes. Contribution analysis is an approach that responds to this challenge.

John Mayne outlines a six step process for undertaking contribution analysis:

  1. An advocacy team identifies the causal relationship it wants to explore: Did a particular set of advocacy activities contribute to a targeted policy change?
  2. An evaluator helps the team describe how they believe their advocacy intervention contributed to the desired policy change and identify the assumptions underlying their story, thus, articulating their theory of change.
  3. The evaluator gathers evidence related to this theory of change.
  4. The evaluator synthesizes the contribution story, noting its strengths and weaknesses.
  5. By gathering perspectives from allied organizations, others involved in the policy change process, and ideally, policy makers themselves, the evaluator tests the advocacy team’s theory of change.
  6. Using triangulation, the evaluator develops a more robust contribution story. With a wide enough range of perspectives collected, this analysis can provide a credible indication of an advocacy intervention’s contribution to a targeted policy change.

Cool Tricks:

  • Timelines can help advocacy teams remember when activities happened and how they relate to each other.
  • Questions such as “And then what happened?” can help a team articulate how an activity contributed to short and medium-term results.
  • Questions such as “What else contributed to that change coming about?” can help a team identify other factors, beyond their activities, that also contributed to the targeted results.
  • When gathering external perspectives, interviewers may start by asking about the targeted policy change and how it came about. Later in the interview, once the interviewee has shared his/her change story, the evaluator can ask about the role of the organization or coalition being evaluated.

Lessons Learned:

  • External stakeholders are more likely to agree to an interview about an initially unnamed organization or coalition if they are familiar with the evaluator. This is especially true with policy makers.
  • Where external stakeholders do not know an evaluator, a well-connected person independent of the organization or coalition being evaluated can facilitate those introductions.
  • Stakeholders will offer distinct perspectives, based on their experience and interests. The more stakeholders one can include, the better.

Rad Resource: APC Week: Claire Hutchings and Kimberly Bowman on Advocacy Impact Evaluation, February 7, 2013.

The American Evaluation Association is celebrating APC TIG Week with our colleagues in the Advocacy and Policy Change Topical Interest Group. The contributions all this week to aea365 come from our AP TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.