Hi evaluators! We are Kate Krueger and Sarya Sok, Evaluation Specialists at the US Department of State Bureau of Democracy, Human Rights and Labor (DRL). Today we are talking about “putting our money where our mouth is” as democracy, rights and governance sector (DRG) evaluators; that is, learning to evaluate more inclusively, equitably, and democratically.
Democracy places sovereignty in the hands of everyday people, and human rights conventions empower them to live according to internationally protected freedoms. Accordingly, we believe that the implementation and evaluation of DRG programming should prioritize the consent and agency of communities in which they operate. This short post shares how we at DRL work to apply this practice to our program monitoring, evaluation, research, and learning (MERL).
Consent: A condition of both healthy democracy and effective DRG programming is the existence of mechanisms by which people can express – or withhold – their consent. Just as an election marred by coercion is not an effective means of obtaining such consent, DRG programs that fail to address the power dynamics between international donor organizations and local communities are likewise compromised.
A more democratic approach to MERL expands the concept of informed consent. Informed consent in power imbalanced relationships is difficult to achieve. A brief pre-data collection notification for local stakeholders who understandably assume that their future financial security may be contingent on evaluation results, cannot elicit meaningful consent. Instead, we advocate that consent to MERL should be built into the design and implementation of DRG programming from the very start. This means that discussion of what success looks like and how it is measured should involve community stakeholders.
Agency: Consent is necessary but not sufficient, for healthy DRG MERL. Community stakeholders need opportunities to exercise agency over the MERL process itself; they should be treated not just as data sources from which to extract information, but as co-creators of the metrics of programmatic success and end users of MERL data.
A more democratic approach to evaluation includes community stakeholders in the development of MEL metrics for two reasons. First, it helps to center the agency and knowledge of communities in a way that many traditional evaluation approaches do not. Second, it helps to yield more precise, and measurable MEL metrics. Community stakeholders are better placed to identify valid proxies for highly abstract intended outcomes, such as “increased civic space” or “more resilient local institutions” far better than outsiders do. Many have their own measures, informed by different epistemological traditions, than traditional international program evaluation approaches. These approaches can uncover a completely different and complementary set of evidence about programmatic results. Evaluators in the Global South have been championing the inclusion of more indigenous MEL approaches and researchers into the international evaluation toolkit for years.
Furthermore, learning and mutual accountability are key to a more democratic approach to evaluation. MERL approaches should exist to protect and inform communities as much as international donors and implementors. A critical – and sometimes missing – factor is the presence of trust and mutually enriching relationships between these groups. Empowering community stakeholders to use MERL data for their own benefit is an essential step toward a more balanced relationship between all groups invested in DRG programming.
Lesson Learned:
We’ve found that incorporating more democratic principles into a MERL practice is largely methods agnostic; an evaluation needn’t be participatory from start to finish, nor does it need to include a particular data collection or analysis approach in order to give community stakeholders the ability to exercise agency over how DRG programming will be assessed.
Rad Resource:
The Everyday Peace Indicators are an inspiring example of community-driven MERL.
The American Evaluation Association is hosting Democracy, Human Rights & Governance TIG Week with our colleagues in the Democracy, Human Rights & Governance Topical Interest Group. The contributions all this week to aea365 come from our DRG TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.