Hello! We are Brian Beachkofski and Jeannie Friedman, Pay for Success (PFS) advisors at Third Sector Capital Partners. We spend most of our time assessing feasibility and designing social sector programs with rigorous evaluations and evidence-based interventions embedded into their contracting structure.
PFS is an innovative contracting model (shown in the figure below) that drives government resources toward high-performing social programs. The PFS model is designed to merge performance measurement using administrative data and rigorous evaluation of long-term outcomes into the contracting structure. This helps ensure that funding is directed toward programs that succeed in measurably improving the lives of people most in need.
Hot Tips:
- Balance Factors in Evaluation Design: A randomized control trial (RCT) was once considered necessary for PFS evaluation, but now it is generally recognized that there is no one-size-fits-all answer. Factors such as operational complexities, sample size, observation windows, budget constraints, and limitations on service providers’ needs should be balanced against each other.
- Focus on Outcomes: Aligning incentives around outcomes is a good first step. Providing interim insight on how the project progresses against those metrics allows the team to make improvements and act on those incentives. This feedback loop is fueled by interim outcome metrics and real-time program delivery modifications. Consistently keeping outcomes in mind maximizes final outcomes for those who are in need. Salt Lake County’s Homes Not Jail illustrates how different evaluation techniques apply to PFS.
- Separate Payment from Policy: An evaluation intending to inform a payment decision is different than one evaluating a policy decision. A PFS project needs to clarify whether the evaluation is to inform payment, with quantifiable impact, or future policy, where causation is paramount. A good example of when we did that was in Santa Clara County’s Project Welcome Home.
- Engage Stakeholders Early: Most PFS projects serve populations with complex, multi-faceted needs that cross multiple government agencies and community partners, which makes defining measurable and meaningful outcomes challenging. Collaboratively refining these goals into defined metrics can gain stakeholder buy-in from all partners.
- Use a Pilot Period: Operationalizing data sharing, referral pathways, and randomization protocols are new skills for many projects. PFS projects are often a government’s first time releasing administrative data outside organizations. Protection requirements and prior practices can make data-sharing feel uncomfortable. A pilot period builds trust and experience in a collaborative shared-data project, easing the full project’s operations.
Rad Resources: Learn more about PFS and projects:
- Introduction to Pay for Success to learn more about how the model works
- Third Sector’s blog, with the latest news and thoughts on PFS are found
- PFS Resource page, with links to more resources
The American Evaluation Association is celebrating Social Impact Measurement Week with our colleagues in the Social Impact Measurement Topical Interest Group. The contributions all this week to aea365 come from our SIM TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.