My name is Allison Van and I am currently an evaluator at the Clinical and Translational Science Awards (CTSA) program at the University of North Carolina in Chapel Hill and the owner of Allison Van Consulting. Previously I managed The Pasture Project for Winrock International, which was an effort to build a movement among farmers in the Midwest to reintegrate livestock rotation both for greater profit and environmental benefit. The project benefitted from funders that were willing to take chances with us, allowing for a budget where resources could be shifted to account for new information or opportunities. Our strategies were highly diverse – demonstration sites on farms, supporting a collaboration of farmer educators, training dedicated farmers in public speaking – yet all directed at influencing the decision-making of individual farmers. Some strategies were about direct influence while others focused on building capacity – in both cases results wouldn’t be seen for years and were highly dependent on external circumstances.
As both the program manager and default evaluator, my goal was to test strategies relatively quickly, rigorously, and cheaply – then modify, end or expand them within 6-18 months. I needed an approach for the team to compare the development of different strategies so money could be funneled where it was most likely to make a difference. Understanding that the core challenge was one of budget allocation amidst uncertainty and long time horizons was critical to selecting the right evaluation approach.
Rad Resources: The combination of Michael Quinn Patton’s Developmental Evaluation and E. Jane Davidson’s Real Evaluation were my constant guides to developing an evaluation approach and making decisions in the context of extreme uncertainty and long time horizons.
Hot Tip: There are profound trade-offs and opportunity costs in social change, making value for money a critical measure of program effectiveness. How programs invest their resources can be the most fundamental determinant of success. A bootstrap method combining cost-effectiveness analysis, the logic model for each strategy, and rubrics of early stage indicators of behavior change allowed us to thoughtfully consider how to make and shift investments.
The American Evaluation Association is celebrating Environmental Program Evaluation TIG Week with our colleagues in the Environmental Program Evaluation Topical Interest Group. The contributions all this week to aea365 come from our EPE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.