Hello, we are your blog series hosts, Barbara Klugman, Heather Britt and Heidi Schaeffer, colleagues of Ricardo Wilson-Grau, the originator of Outcome Harvesting. Ricardo mentored and inspired many members of the AEA community. Sadly, he passed away on December 31, 2018. This series of posts on Outcome Harvesting is in his honor. In this first blog we use his own words to introduce Outcome Harvesting (OH).
“Outcome Harvesting is designed for grant makers, managers, and evaluators who commission, manage or evaluate projects, programs, or organizations that experience continual change and contend with unexpected and unforeseeable actors and factors in their programming environments.”
“Unlike other monitoring and evaluation approaches, Outcome Harvesting does not necessarily measure progress towards predetermined objectives or outcomes, but rather, collects evidence of what has changed and then, working backwards, determines whether and how an intervention contributed to these changes.” (2019, p1)
OH is an appropriate method when the evaluation is asking “who changed and what changed?” It is not the right method for evaluation questions such as: “Did the training program increase participants’ knowledge and skills?” It is a good method for asking questions such as: “What are the participants doing differently after acquiring new knowledge and skills?” And, “What do the organizational policy and/or practice changes look like since the training program began?”
OH describes an outcome as an observable change in behavior (relationships, actions, activities, practices or policies) of an individual, group, community, organization in civil society, corporation, government, media, or member of public. In every outcome harvest, the intended users of the harvest findings define what constitutes an outcome. An OH should seek outcomes that relate to the evaluation question, but also note unintended or unexpected outcomes and both positive and negative outcomes.
Outcome Harvesting follows six interactive steps:
An outcome description includes:
- summary: who, when and where changed their behavior;
- contribution: how the intervention contributed, directly or indirectly, towards influencing that change in behavior;
- significance: of the outcome in relation to the organization’s or initiative’s goals.
Harvesters compile and categorize outcome descriptions (e.g. by type of actor, location, or other useful grouping). Then, harvesters interpret the patterns in the aggregated outcomes to answer monitoring and evaluation questions. Harvest users should be deeply engaged throughout analysis and interpretation.
By the end of 2016, Outcome Harvesting had been used by over 400 networks and associations, NGOs, community-based organizations, research institutes, and government agencies in 143 countries on all seven continents.
Hot Tip: Ensure that the initiative is at the point of influencing outcomes, as it can be inappropriate and disempowering to do an OH too early.
Lesson Learned:
It takes practice to confidently and accurately identify and draft outcomes and to engage the users in the whole process. If using OH for the first time, consider working with a co-facilitator or mentor familiar with the method.
Rad Resources:
- Outcome Harvesting: Principles, Steps and Evaluation Applications (IAP, 2018) by Ricardo Wilson-Grau
- World Bank, Outcome-based Learning Field Guide (2014)
- Outcome Harvesting website
The American Evaluation Association is celebrating Outcome Harvesting week. The contributions all this week to aea365 come from colleagues of the late Ricardo Wilson-Grau, originator of Outcome Harvesting, and these articles are written in his honor. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Pingback: OM and OH – OPC program design training