“But wait — there’s more!” Hi! I’m Brian Yates. Yes, this is the third piece in a series of “AEA365’s” on using cost data in evaluation … and not entirely inappropriate for your former AEA Treasurer to author. (I’m also Professor in the Department of Psychology at American University in Washington, DC.)
My past two 365ers (you can find them here and here) focused on evaluating costs of resources consumed by programs, and the monetary and monetizable outcomes produced by programs. With those two types of data, we can begin evaluation of the programs’ cost-benefit.
Lesson Learned – Funders should ask “Is it worth it?” and not just “How much does it cost?” The impulse to cut programs to meet budget goals actually can increase costs and bust budgets if the programs would have increased employment (and taxes paid), reduced consumers’ use of other services, or both. Evaluators can investigate this by comparing program costs to program benefits.
Hot Tip – Strategies for quantifying Value. “Is it worth it?” can be answered in different ways, including:
- dividing benefits by costs (benefits/costs ratio),
- subtracting costs from benefits (net benefit), and
- measuring the time required before benefits exceed costs (Time to Return On Investment).
Each cost-benefit index describes a different aspect of a program’s cost-benefit relationship, and all can be reported in a Cost-Benefit Analysis or CBA.
Hot Tip Too – Costs, benefits, and cost-benefit relationships can be measured at several levels of specificity (such as for an individual in a program or for a new implementation of the program). Cost-benefit differences between programs as well as between consumers in the same program are important to understand, too.
Lesson Learned – Programs whose costs exceed measurable benefits still can be funds-worthy. A “news” story in humorous Daily Onion a few years back made an important point with its story, “Cost of living now outweighs benefits” (http://www.theonion.com/articles/cost-of-living-now-outweighs-benefits,1316/). Few of us would be inclined to act in a summative manner based on these “findings” (even if they were valid)! So may the value of many programs be underestimated or missed entirely when viewed through the “green eyeshade” of an exclusively pecuniary perspective.
Also, some programs provide services to which all people are legally entitled. Evaluations of these can instead ask, “What is the best way to deliver the highest quality services to the most people for the least money?”
Next time: how to include costs when evaluating nonmonetary program outcomes.
Resources: Text: Michael F. Drummond’s (2005) Methods for the Economic Evaluation of Health Care Programmes.
CBA for environmental interventions
Want to learn more? Register for Evaluating and Improving Cost, Cost-Effectiveness, and Cost-Benefit at Evaluation 2014.
This week, we’re featuring posts by people who will be presenting Professional Development workshops at Evaluation 2014 in Denver, CO. Click here for a complete listing of Professional Development workshops offered at Evaluation 2014. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Great points, and I love the Onion article! Measuring time to return on investment is especially important for many social programs and systems change initiatives that have higher costs as they ramp up but lead to savings in the long run.