My name is Nicola Dawkins, and I am a Principal with ICF Macro. As an international evaluation, research, and management consulting firm, our work focuses on assessing and strengthening health and other social programs and policies for an array of governmental clients.
As evaluators, I know we are acutely familiar with the limited resources generally available to conduct large scale, rigorous outcomes-focused evaluations of initiatives. As individuals, I suspect many of us also recognize such large scale evaluative investments are not appropriate for every initiative—perhaps many of those we have seen. Nevertheless, accountability remains key, and investments in widespread programming still need to be guided by data about what works. How can we advise our clients to make such investments wisely? Here are a couple of tools to add to your evaluator’s toolkit.
Hot Tip: Try implementing evaluability assessments (EAs) prior to committing to a large scale outcome evaluation design. Popularized by Joseph Wholey in the late 1970s, the approach is making its way back into the limelight—and with good reason. With a relatively modest outlay of resources, EAs can provide strong insight regarding an initiative’s design, implementation, and readiness for an outcomes-focused evaluation.
Hot Tip: Fortunate to be working in an area with multiple potentially promising initiatives and little current scientific evidence of effectiveness to guide replication? Try applying the Systematic Screening and Assessment (SSA) Method. Pairing evaluability assessment with expert review and judgment, the SSA Method can generate a wealth of knowledge about a series of initiatives with modest resource investment.
Rad Resource: Learn more about applying EAs as part of the SSA Method in the latest New Directions for Evaluation. Check out the overview article: Leviton, L. C., & Gutman, M. A. (2010). Overview and rationale for the Systematic Screening and Assessment Method. In L. C. Leviton, L. Kettel Khan, & N. Dawkins (Eds.), The Systematic Screening and Assessment Method: Finding innovations worth evaluating. New Directions for Evaluation, 125, 7–31.
In the issue we present the method and offer an example applied to obesity prevention initiatives. Give it a whirl. You might find you’ve read the entire issue before you know it!
Hot Tip: AEA members have access to all articles in New Directions for Evaluation as a benefit of membership. Sign on to the AEA website at http://eval.org/ with your AEA username and password. Not a member? Join AEA via the online forms at http://www.eval.org/membership.asp.
Want to learn more about evaluability assessment? Attend Nicola’s workshop at the AEA/CDC Summer Evaluation Institute next week in Atlanta, GA http://www.eval.org/SummerInstitute10/default.asp