Greetings all! I am Agata Jose-Ivanina and I am a Senior Associate with ICF International, a research and evaluation firm headquartered near Washington, DC. As the budget debate has been heating up and educational agencies have to make hard choices, we have seen an increased demand to include a cost component as part of program evaluation. When trying to measure and analyze costs, evaluators should follow four steps:
In Step 1, you need to identify all of the possible costs of a program. Using the “ingredients method,” list out all of the possible cost categories of a program, as well as the individual costs within each category. For example, an evaluation of an educational technology program might include costs for personnel, hardware, software, training, troubleshooting, and maintenance.
Hot Tip: When identifying cost ingredients, it is a good idea to work together with the program staff who know the most about the program’s operations.
In Step 2, you develop a calculation for each of the individual costs identified in Step 1. For example, to estimate the value of the time that teachers spent in training, you would multiply the length of the training by the number of teachers by the hourly stipend that teachers receive for participating in professional development.
Hot tip: In this step, write out a formula for each calculation. Writing out all formulae will help you understand what data you need to collect and all the assumptions you may need to make.
In Step 3, you collect each of the individual pieces of data that is necessary for Step 2 calculations. Evaluators can’t rely solely on program budgets when collecting cost data. Budgets often bundle costs of several programs together, and it may be hard to isolate the information that you need. Moreover, budgets may not include items that were not explicitly paid for, such as the cost of office space or time from salaried employees.
Hot Tip: Receipts, interviews, market research, and surveys — a researcher may need to employ all these strategies to get at the total cost of the program.
Step 4: Once you have your data, it is just a matter of plugging them into the calculations you developed in Step 2, and adding them up to get a total program cost.
Rad Resource: If you are new to cost-benefit or cost-effectiveness analysis, the book to start with is Levin and McEwan’s Cost-Effectiveness Analysis: Methods and Applications. Not only does it explain the concepts very well, it has a wonderful bibliography.
This contribution is from the aea365 Tip-a-Day Alerts, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to firstname.lastname@example.org. Want to learn more from Agata? She’ll be presenting as part of the Evaluation 2011 Conference Program, November 2-5 in Anaheim, California.