Hi, my name is Jeanette Tocol and I am a Monitoring, Evaluation, and Learning Advisor at the American Bar Association Rule of Law Initiative (ABA ROLI). One of my primary roles at ABA ROLI is to oversee evaluations. I find that many of the evaluations we commission focus on establishing the linkage between capacity building and policy changes in the democracy, rights and governance sector.
Let’s face it: drawing a straight line from capacity building to policy change is near impossible (if not unrealistic). So, what is the next best way to determine evaluation scope and focus in on efforts that lean towards establishing the connection of capacity building to policy change? We found that applying specific frameworks to be useful for us to develop evaluation questions that establish the relationship between results and the process of change.
Many programs are framed to first affect individual knowledge and skills, as well as behavior; and expect that in time this will impact organizational performance and add value for the policy area. One useful framework for evaluating such changes is the Kirkpatrick Model for Training Evaluation which links reactions to training to learning, behavioral change and results. When generalized to other aspects of capacity development, the Kirkpatrick Model supports measurable logic models from individual to organizational changes.
Some programs treat capacity building as “technical assistance” to institutions. This requires a framework capable of assessing the results chain from inputs, to outputs/levers of change, to institutional outcomes. In turn, this encompasses assessing the areas of performance, adaptability, and stability, before assessing impact to national policy or development goals.
Some programs provide for interlinked domains of capacity where “key populations” of the community need strong organization to act on behalf of a community, such that this key population is the focus of capacity building efforts along with government as “duty bearers”; and their interactions are expected to lead to policy changes that are intrinsically inclusive and participatory. The Collective Impact Model has been useful in various efforts where community capacity is seen not only for the output and outcomes it may produce, but that the value of a well-drawn process is as important.
Lessons Learned
- Capacity building is a marathon and not a sprint. Thus, monitoring, evaluation and learning practices may best be set in intervals, with regular checks or reflection activities set for process, results, and context. In order to accomplish regular checks, MEL activities must be incorporated in work plans.
- Democracy, Rights and Governance (DRG) outcomes and impacts are often long term commitments. Setting cost-effective, timely, accessible, and reliable proxy indicators may be the next best way to assessing whether these long term outcomes or results can be achieved at a later time – a time after development programs.
- Clarity on what we mean by “capacity building” and its connection to the desired changes in policy outcomes is obviously needed. Nonetheless, funders as well as development practitioners remain vague in defining what they really mean. Clear frameworks and theories of change help define the desired change process and eventually measure against results.
Rad Resources
- Development of Framework for Evaluating Capacity Development Initiatives in International Development by Korvojs for examples of frameworks models
- Measuring Capacity by UNDP for examples of measuring capacity building
- Learning in Action: Evaluating Collective Impact by Parkhurst and Preskill for examples of evaluating collective impact
- The Proxy Challenge: Why bespoke proxy indicators can help solve the anti-corruption measurement problem
The American Evaluation Association is hosting Democracy, Human Rights & Governance TIG Week with our colleagues in the Democracy, Human Rights & Governance Topical Interest Group. The contributions all this week to aea365 come from our DRG TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.