My name is Sandi Adams, and I am the Program Associate in Evaluation and Planning at the William Penn Foundation. The William Penn Foundation is a regional foundation dedicated to improving the quality of life in the Greater Philadelphia region through efforts that foster rich cultural expression, strengthen children’s futures, and deepen connections to nature and community.
A key role of the philanthropic sector is support for the development and testing of innovative responses to challenges constituents face. The intent is to establish successful models for systems change that public systems — such as state or municipal governments or local school districts — will adopt. Successful social programs often fail though when widely replicated because adequate attention is not paid in the design, pilot implementation, and evaluation stages to both programmatic and systemic elements critical for replication and scalability.
This has increasingly important implications for the philanthropic sector and how we participate in and monitor design, implementation, and evaluation of pilot programs, but also for evaluators in how they design evaluations of pilot programs.
Hot Tip: Evaluation (in addition to the development and implementation) of a demonstration program should consider four key elements:
- Ability to demonstrate outcomes: Program effect.
- Replicability: Whether the defined set of measurable services and products offered can be implemented with fidelity in various settings (e.g., schools) with different contexts (e.g., demographic constituents) and personalities (e.g., teachers) to achieve anticipated results.
- Scalability: Whether the cost of implementing the program is reasonable in relation to the benefit it is expected to produce.
- Scalability: Level of interest in the program by the target of change, or the individual or entity in the position to facilitate a change in the system. In order to change public systems, it is essential to understand the underlying politics of the situation and the lifecycle of systems change, including whether an intervention proven to be successful will have contagion. In most cases, this results from the target of change’s commitment and capacity (e.g., finances) to implement (e.g., scale) the intervention.
Rad Resource: Harvard Family Research Project’s The Evaluation Exchange is a newsletter on emerging strategies in evaluation. The spring 2010 issue is focused on scaling impact and includes articles on the role evaluation can play: http://www.hfrp.org/var/hfrp/storage/original/application/b09c074129f1943b4a172d23fb8542b5.pdf
The American Evaluation Association is celebrating evaluation in Not For Profits & Foundations (NPF) week with our colleagues in the NPF Topical Interest Group. The contributions all this week to AEA365 will come from our NPF members and you may wish to consider subscribing to our weekly headlines and resources list where we’ll be highlighting NPF resources.