How do you as an evaluator learn how individuals or communities experience an investment intended to benefit them when the intervention is not directly with them? This situation faced us – Jane Reisman, Jasmine Johnson, and John Sherman (aka JCrew) – in our evaluation of the Community Investment Guarantee Pool (CIGP). This pool of philanthropic and mission-oriented investors uses an innovative social finance tool known as an “unfunded guarantee” to test credit risk assumptions that historically disadvantaged businesses led by people of color or women. The pool targets small businesses, affordable housing developers, and climate solution providers.
LOCUS, CIGP’s fund manager, issues guarantees to community development lenders and other intermediaries who work with borrowers perceived as uncreditworthy by conventional finance standards. Functioning like an insurance policy, the guarantees reduce the real or perceived riskiness of a loan, thus enabling lenders to issue loans that do not meet the standard credit criteria.
Our job? Prepare and implement a developmental evaluation that captures learnings of the investors, LOCUS, the intermediary lenders receiving the guarantees, and the borrowers.. We collected information through peer learning sessions, interviews, surveys, and written reports with all of the actors except the borrowers. Spread across rural and urban communities in over 20 states, the borrowers only have a direct relationship with their lenders whereas LOCUS, the intermediary lenders and investors frequently engage with one another.
Lack of access means that borrower’s voice and experiences cannot inform the developmental evaluation and thus, cannot help the other actors learn how to collectively better serve the borrowers. To work around this impediment, we adapted the Most Significant Change story (MSC) methodology originally articulated in 1994 by Rick Davies in an international development context.
Working with the intermediary lenders, we ask them to identify two or three borrowers that experienced significant changes in their businesses due to the loans, and to share those stories with us via an MSC template we created. Unlike the established MSC approach, due to lack of access to the borrowers, we did not develop anticipated domains or outcomes of change with them; instead we identify the domains of change from those shared stories.
Guided by the domains of change emerging from the stories, we then select two MSC stories from all submitted for a more in-depth study to gain further insights. We completed one round of MSCs in 2022 and are repeating it again in 2023. To develop the two deep dive MSCs, we will conduct independent interviews with the lenders and borrowers. Respecting their time, both lender and borrower are offered an honorarium, $1,000 for the lender and $500 for the borrower.
Lessons Learned
So, what is the importance and contribution of the MSC approach and stories for CIGP? As the evaluators, it brings to life the challenges in accessing affordable capital among business owners and affordable housing developers led by people of color and women, as well as climate solution providers focused on lower income communities. It also shines the light on what happens for them, their employees, and communities when they can get funding. Moreover, it hones the types of learning questions we explore with the investors, LOCUS, and the intermediary lenders; questions that help inform and collectively revise CIGP’s theory of change.
The benefits extend beyond informing the developmental evaluation. It also provides well-vetted, useful stories for the funders, LOCUS and the lenders about the impacts of CIGP on credit-challenged businesses and projects that, if given the financial opportunity, can likely succeed.
The American Evaluation Association is hosting Social Impact Measurement TIG Week with our colleagues in the Social Impact Measurement Topical Interest Group. The contributions all this week to AEA365 come from our SIM TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.
Fabulous. Keep these MSC and other Participatory DIM&E models and examples coming.