Hi, I am Barbara Klugman, an independent evaluator based in South Africa. I use outcomes harvesting mostly for evaluating progress of social justice advocacy initiatives and outcomes of training initiatives. I also support funders in using it for monitoring progress and strategic learning about their grantmaking portfolios.
OH can reveal processes and patterns of change over time by analyzing sets of outcomes. Ricardo Wilson-Grau, the originator of OH, notes that, “The true value of an outcome harvest is not collecting individual outcomes but demonstrating how sets of outcomes reveal processes and patterns of change over time. So, the evaluator must take care not to be drowned in outcome detail and ensure that the story or picture of change emerges.” [Outcome Harvesting Principles in Practice 2016] This can be done in diverse ways.
Cool Trick: Collect and analyze small outcomes over time. When collecting outcomes over time, OH tells the story of how one group, in collaboration with others, influenced specific outcomes over time as illustrated by the example below.
Cool Trick: Categorize outcomes by type. During analysis, categorize the outcomes into different types. For each harvest, the types are developed based on the question being asked. For example, a harvest user tracking outcome of an advocacy initiative sorted outcomes as illustrated in the figure below. Analysis revealed the proportions of outcomes aimed at influencing government that were actual changes of policy versus changes in narrative and debates, versus changes in organizational capacities, among others, again something unpredictable in advance and across very diverse organizations and regions. It is this kind of quantitative data that gives an overview of what is happening in a large multi-issue, multi-organizational initiative and surfaces strategic questions going forward.
Cool Trick: Look for patterns. OH reveals patterns that of change not predicted in advance. For example, an outcome harvest was part of an evaluation of Ford Foundation’s 2012-2017, $54million Strengthening Human Rights Worldwide global initiative (SHRW). The initiative aimed to influence the human rights movement internationally, in particular the agency and influence of groups in the global south. The grantees developed their own strategies to achieve this broad objective. Not surprisingly, the actual outcomes targeting governments or inter-governmental human rights bodies demonstrated most changes were at the international level (42%) or at national level (35%). What was unanticipated was that 23% related to regional human rights institutions. This unintended and unexpected finding alerted the evaluation users to the extent to which human rights groups are now focusing attention at regional level.
Lessons Learned: Harvesting over a number of years reveals the nuances of organizations’ contributions towards high level goals. Harvesting across a diverse portfolio of organizations demonstrates their cumulative influence on different types of outcomes.
Rad Resource:
- Further demonstration of how the method reveals patterns can be seen in the report of the Ford Foundation Review
The American Evaluation Association is celebrating Outcome Harvesting week. The contributions all this week to aea365 come from colleagues of the late Ricardo Wilson-Grau, originator of Outcome Harvesting, and these articles are written in his honor. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.