My name is Kathryn Hill and I work with the state of Minnesota’s GEAR UP college access initiative as the Evaluation and Research Manager. GEAR UP is the acronym for the Gaining Early Awareness and Readiness for Undergraduate Programs federal grant program, designed to increase the number of low-income students who are prepared to enter and succeed in postsecondary education. GEAR UP provides six-year grants to states and partnerships to provide services at high-poverty middle and high schools.
I collaborated with external evaluators to develop a framework for a cost-benefit study. I learned a lot from this evaluation management experience, and I have compiled a few tips.
Hot Tip- Start with a thorough literature review: To develop a cost analysis framework, you need evidence to support proposed outcomes and to attribute economic value for these proposed outcomes. It is possible to rely on existing research to determine a projected “effect” for the program when building a framework, but you should have results from your own rigorous evaluation before proceeding with the cost study.
Hot Tip- Use evidence to articulate the program theory: A logic model is a common starting point for communicating program theory, and it is essential for cost analysis.Using program theory, an evaluator can move toward a clear identification of the different program components. These program components guide the economic calculations of “inputs”. Financial calculations of program expenses are viewed from program staff and participant perspectives. Both are important for determining the cost of a program. If it takes a lot of time for a staff member to develop/deliver a program component that is utilized by only a small number of students, the cost per participant will be high for that specific component. This brings us back to the “effect” issue, because you may want to know what proportion can be attributed to each of your program components.
Hot Tip- Find a way to document/describe what is actually happening in the program: The external evaluators developed an interactive format for interviewing all program staff. Staff members found the process interesting; some even thought it was fun!
Hot Tip- Think “program components” rather than “accounting categories” when recording expenses: For example, our program provides college visits, and the cost includes transportation expenses. However, that budget category usually has transportation expenses for EVERYTHING, including field trips, summer programs, etc. You will save yourself many headaches if you set up detailed sub-codes for expense records.
This aea365 contribution is part of College Access Programs week sponsored by AEA’s College Access Programs Topical Interest Group. Be sure to subscribe to AEA’s Headlines and Resources weekly update in order to tap into great CAP resources! And, check out the CAP Sponsored Sessions on the program for Evaluation 2010, November 10-13 in San Antonio, to learn more from Kathryn.