Hello! We are Robin Kane of RK Evaluation and Strategies, Carlisle Levine of BLE Solutions, LLC, Carlyn Orians of ORS Impact, and Claire Reinelt, independent evaluation consultant. We offer evaluation, applied research and technology services to help organizations increase their effectiveness and contribute to better outcomes.
In our advocacy and policy change evaluation work, we have found contribution analysis useful for identifying possible causal linkages, and determining the strength and likelihood of the causal connection.
Contribution analysis starts with working with advocates to develop a theory of change describing how they believe a specific change came about. The evaluator then identifies and tests alternative explanations to that theory of change by reviewing documents and interviewing advocates’ allies, others trying to influence a policy change, and policy makers themselves. Then the evaluator writes a story outlining the advocates’ contribution to a specific change of interest, acknowledging the roles played by other actors and factors.
When trying to identify possible causal linkages in advocacy and policy change evaluation, why choose contribution analysis?
- Contribution analysis is a good choice when the need for information emphasizes plausible demonstration of credible contribution over proof or quantification of contribution.
- Often in an advocacy process, multiple stakeholders are involved. Contribution analysis provides a method for distinguishing among contributions towards a policy change.
- Contribution analysis allows for the acknowledgement of the contributions of different actors and factors to a policy change.
- Through testing alternative explanations, contribution analysis offers a rigorous way to assess what difference a particular intervention made.
- Contribution analysis was developed as a performance management tool, and works especially well when performance outcomes and benchmarks are clear. In advocacy evaluation, goals and strategies adapt and respond to the political environment. To address this challenge, we developed timelines of actions, including high-level policy meetings, communications and media efforts, research, and policy briefs and position papers. We mapped our timelines to strategic moments when there were incremental changes related to our policy of interest. We could then trace how an advocacy effort influenced and was influenced by a policy change process.
- Interpreting information received can be tricky, since different stakeholders have not only different perspectives regarding how change came about, but also different interests in how that change is portrayed. Being aware of stakeholders’ perspectives and interests is critical for interpreting the data they provide accurately.
Rad Resources: Stay tuned for our brief on using contribution analysis in advocacy and policy change evaluation; available prior to AEA 2017 on our websites and also on www.evaluationinnovation.org
The American Evaluation Association is celebrating APC TIG Week with our colleagues in the Advocacy and Policy Change Topical Interest Group. The contributions all this week to aea365 come from our AP TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to firstname.lastname@example.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.