APC Week: Anna Williams on When is a policy “win” real? Why does this matter when evaluating policy advocacy?

Hi. I’m Anna Williams, Senior Associate at Ross & Associates Environmental Consulting, in Seattle, Washington.

Advocates, their funders, and policy advocacy evaluators seek to understand the results of policy advocacy work. Advocates promote the adoption (or reversal) of government policies, and many use the term “wins” to refer to successful milestones in their advocacy work. However, this term is often undefined and lacks context. “Wins” may mean many things: endorsements from public figures, favorable policy proposals, government bodies voting favorably, passage of desired policies into law, etc. Contribution/attribution aside, upon examination, the term “win” may or may not have a meaningful relationship to actual policy change.

Lessons Learned – The stark reality: Policy change is typically not linear; and it’s a long-term endeavor. The work can be downright messy. Progress one year can be weakened or reversed the next. Some policies are very weak by the time they are passed; others may have unforeseen consequences or fatal flaws. Later, implementation may be anything but guaranteed. Context matters. Policy work varies from place to place, country to country, venue to venue: one size does not fit all. There are windows of opportunity during which significant and durable policy change can occur quickly; however, these are the exceptions.

When parties claim policy “wins” we could ask for more precision. One philanthropy I work with has moved from “win” to “policy adoption” and defines the latter as follows: “Decision-makers have adopted, approved, or otherwise agreed to the policy or action; implementation is not yet underway.” (This philanthropy also defines stages preceding and following “policy adoption,” while acknowledging the limitations of this linear framework.)

A clearer view on policy (and advocacy) progress and “wins” can have a sobering effect, especially when we acknowledge the slow pace and volatility of policy change. But we need to help funders be realistic about the long-term nature of policy advocacy work, and avoid illusions concerning return on investment. The advocacy community need not be apologetic about these realities; however, it may take time to close the delta between funder expectations and on-the-ground realities. We all need to tell funders what they need to hear – not what they want to hear.

Lessons Learned: As policy advocacy evaluators, we should encourage advocates, policy advocacy funders, and the evaluation community to be clear about “wins” and to unapologetically convey that, even under ideal advocacy conditions, policy change takes time and even then can be vulnerable.

How do others view this issue? How do others define and track policy progress? What have others experienced when having these kinds of discussions with advocates and their funders?

We’re celebrating Advocacy and Policy Change week with our colleagues in the APC Topical Interest Group. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

1 thought on “APC Week: Anna Williams on When is a policy “win” real? Why does this matter when evaluating policy advocacy?”

  1. Anna, I think you make several excellent points here. For NCRP’s series of 7 regional reports on the impacts of advocacy and community organizing, we tried to distinguish between outcomes (what happened) and impacts (what the effect was). For example, in a state campaign to raise the minimum wage, an outcome would be that the minimum wage legislation was passed in the legislature and signed into law. The impact was that low-wage workers’ incomes were raised as a result of the minimum wage increase. To the extent possible, our researchers tried to verify that the policy changes documented by the nonprofits we studied happened and were implemented. In one case, a law had been passed to improve working conditions for day laborers, but the state labor department had done nothing to implement it, so we didn’t include it as an impact.

    In addition to emphasizing the long-term nature of advocacy campaigns and the value of interim benchmarks along the way, we also made the point repeatedly that implementation and monitoring (especially against threats of repeal) are critical roles that nonprofits often need to play, to help grantmakers understand that the work to ensure the intended benefits of a policy are realized must be supported, as well as the initial effort to help pass a policy.

    A summary of our research on advocacy by 110 organizations across 13 states will be available on our website toward the end of next week, along with a searchable database of all of their advocacy impacts, at http://www.ncrp.org/gcip.

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.