Thank you for subscribing to aea365! AEA is aware of the issue impacting email receipt of aea365 blog posts. Our RSS Feed provider has announced a reduction in maintenance which has unfortunately resulted in sporadic email receipt of daily posts. AEA is transitioning to a new provider and intend to roll out a new, improved service in early September 2022! Until then, please continue to regularly check back to aea365 for daily tips, tricks, and resources for evaluators by evaluators. You may also add AEA365 blog’s feed URL – https://aea365.org/blog/feed/ – to your favorite RSS! Stay tuned for additional details! For questions, please contact AEA at info@eval.org.
Hello, AEA – we are Prakhar Sharma and Keith Proctor of the International Republican Institute (IRI), a nonprofit, nonpartisan organization that supports democracy worldwide.
In countries with fragile democratic institutions, governance is routinely undermined by a lack of credible information about the public’s priorities and preferences. This can undermine government-citizen interactions, the responsiveness of elected leaders, and popular confidence in representative government. Too often surveys are seen as useful only for message framing rather than a democracy building program component in their own right. To fill this gap, IRI has long conducted public opinion research – including more than a thousand polls in 107 countries over the last 23 years – to put citizen priorities front-and-center before policymakers and political parties.
With support from the National Endowment for Democracy, we evaluated IRI political party projects that utilize public opinion polling. Through a series of semi-structured interviews with political party members and country team colleagues, we evaluated programs in four countries: Bangladesh, Moldova, Turkey, and Ukraine. These cases represent different regional contexts and party systems, variation in the scope of IRI’s engagement, and divergence in the degree of closing space contexts.
We wanted to explore several questions. To what extent do political leaders use public opinion polling to inform policy positions or outreach? How does polling help parties become more issue-focused? How do contextual differences, including variations in the party structure or political system, influence the uptake of polling data?
Lessons Learned
Tailor outreach and polling activity design to increase engagement and generate buy-in.
While polling activities are more likely to gain traction among party members who have the capacity and incentive to engage public opinion data, greater program impacts may be realized by reaching participants who are less data savvy and/or more resistant to polling. To navigate this tradeoff, implementers should tailor outreach and activity design to specific audiences. Consider separate activities with more intensive content for data-savvy partners, and more “introductory” presentations for non-specialists. Finally, identify champions for data-informed approaches, who might be willing and able to generate buy-in among more skeptical co-partisans.
Integrate partnership-building and network-strengthening into theories of change.
Across contexts, providing transparent, credible polling data helped IRI programs build trust and relationships with political parties. Utilizing polls to facilitate partnerships was often an unstated (and therefore unmeasured) goal of implementation. Programs intending to use public opinion polls to convene networks, maintain relationships, and engage new stakeholders should consider building this explicitly into theories of change, along with monitoring plans to capture these important outcomes.
Incorporate objective measures to evaluate policy attitudes and platform changes.
Attributing outcomes to program activities is difficult and requires careful attention to evaluation design and data collection. The reliance on self-reported perceptions of outcomes – such as a change in policy, platform, or coalition strategy – can be particularly thorny, given that respondents may be incentivized to overestimate program impacts. We recommend adopting relatively objective measures of participant policy attitudes or positions (e.g., via surveys) or changes in policy platforms (e.g., through qualitative content analysis, or analysis of party administrative data, such as party meeting minutes, agendas, transcripts) to better evaluate impact.
Ultimately, programs are more likely to achieve impact when design is informed by robust evaluation and research rooted in the local operating context, which can help us refine what we are trying to achieve, who we are trying to reach, and how we will measure impact. As we work to further integrate evidence into program design, careful examination of local factors remains a vital starting point.
The American Evaluation Association is hosting Democracy, Human Rights & Governance TIG Week with our colleagues in the Democracy, Human Rights & Governance Topical Interest Group. The contributions all this week to aea365 come from our DRG TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.
That was a mouthful, but thanks for a deep dive into this one