We are Debby Nixon Williams and Alyna Wyatt (Social Impact Measurement specialists at economics-based consulting firm Genesis Analytics) and Alisee de Tonnac, of Seedstars Global, an Impact Investor and business incubator supporting high-growth entrepreneurs in emerging economies.
Over the last few months, we have been working
together to develop a Theory of Change, corresponding indicators, and a
practical and action-oriented performance framework that will be useful for
Seedstars Global as an impact-led business to track, measure, and assess social
impact. We have moved mountains within the organization in terms of progressing
a culture of evaluative thinking and diligent measurement. Seedstars is a
social enterprise that has developed in the private sector sphere with little
exposure to the development sector’s best practice models or language on impact
measurement. As Seedstars began to expand its operations and seek partnerships
with development organizations, they were unable to translate their business
impact into a concise representation of social impact that resonates with the
development sector. This challenge to communicate impact also led to confusion
within Seedstars regarding impact intentions for new hires. Bridging the gap
between the business lens and the social impact lens is a tricky dynamic, and
we could have ended up with a completely useless engagement, but a few specific
things we did may help others bridge gaps in future engagements:
Lessons Learned:
- Despite early skepticism that a Theory of Change and social impact-focused performance framework would be valuable for Seedstars Global, both sides actively committed to learn about the other’s context and needs. Alisee patiently explained the business model while we listened and sought to understand how impact is achieved throughout the interventions. We listened to each other!
- Seedstars tested the measurement potential of indicators by pulling together a centralized historic representation of impact for as many of the indicators as possible. This led to useful iterations in the definition and grouping of indicators. The testing process helped Seedstars to operationalize their measurement system by creating a shared Google tool with individuals assigned responsibility for reporting specific indicators.
Rad Resource:
- Genesis used the Impact Management Project framework as the approach for the measurement system to bridge the gap between what Seedstars experiences as an impact-oriented organization in the private sector and development framing of results. This framework was a valuable grounding tool for Seedstars’ approach to measurement.
We have all learned a great deal throughout our engagement and have a newfound respect and understanding that impact can be achieved in many different ways!
The American Evaluation Association is celebrating Social Impact Measurement Week with our colleagues in the Social Impact Measurement Topical Interest Group. The contributions all this week to aea365 come from our SIM TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on theaea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by theAmerican Evaluation Association and provides a Tip-a-Day by and for evaluators.
Pingback: #EvalTuesdayTip: Impact Investing and Evaluation - Khulisa
Business Sharks and Wall Flowers Can Be Friends: Evaluation in the Business World
Response to post: SIM TIG Week: Listening To Each Other: Achieving Impact Through The Private Sector In Emerging Markets by Debby Nixon Williams, Alyna Wyatt and Alisee de Tonnac / Social Impact Measurement / By Sara Vaca / November 6, 2019
https://aea365.org/blog/sim-tig-week-listening-to-each-other-achieving-impact-through-the-private-sector-in-emerging-markets-by-debby-nixon-williams-alyna-wyatt-and-alisee-de-tonnac/
The post titled “Listening To Each Other: Achieving Impact Through The Private Sector In Emerging Markets” sparked my attention because I was excited to see someone finally speak out about the business applications of program evaluation! In this post, I see that I am not alone in my feelings that there seems to be a disconnect between our perceptions of business evaluations and program evaluations. In the post, the author brings to light something I have felt somewhat of a disconnect on: “Despite early skepticism that a Theory of Change and social impact-focused performance framework would be valuable for Seedstars Global, both sides actively committed to learn about the other’s context and needs. Alisee patiently explained the business model while we listened and sought to understand how impact is achieved throughout the interventions. We listened to each other!” This statement verified that I was not alone in my thoughts and that the general evaluation community and business community come from such different standpoints that there is no way that they could connect on a meaningful level. Clearly, that is not the case. I can most definitely validate that.
After completing the following course from Queen’s Faculty of Education: Program Inquiry and Evaluation, I can assure you all that business and program evaluations come from the same place. They are both rooted in 3 pillars: deeper understanding, the desire to improve, and the need to track progress towards goals. The diction used to describe these processes in business environments may seem intimidating… “performance reviews”, “funding cuts” or “ROI (return on investment) are terms that often triggers feelings of apprehension for anyone who hears it. It is no different (in terms of the end goal), than what occurs at the program evaluator’s desk in my opinion. The diction used to describe some of these “business shark” like terms are simply put, more flowery in the evaluator’s handbook, hence the passive term I used in the title… “wall flower”. Performance reviews become program theories and data collection, funding cuts become proper allocation of resources and ROIs become feasibility standards, practical procedures and fiscal responsibility.
Throughout this course, I have learned much about evaluation, not only through my own eyes butt rough the eyes of my peers. I read about what evaluations can do for many different communities, social entities and educational programs. However, I have been constantly thinking about the business applications of evaluation throughout the course. As I reflect back on to my introduction to the course community, I see that what I have been doing at work truly was evaluation all along, even though I thought it might not be. If I can sum up my biggest take away from this course and how I will implement it into my practice as a Denturist, I would say I feel far more confident in my ability to create and comprehend program evaluation designs. I feel as if the bulk of the work in evaluating a program comes back to the evaluation design itself. Truly understanding the goals and how these are executed, and how they could be assessed was the biggest challenge for me. It seemed so simple when I read about it, but it became more and more evident throughout the course that there are so many methods to verify questions or evaluate hypotheses about the program, along with a myriad of confounding variables. I believe this understanding will make me a better employer and educator as I can better determine how employees are progressing towards goals in a comprehensive manner. Here is a short list of how I can see program evaluation becoming an integral part of my practice:
Determining use of equipment to meet patient needs
Evaluate return on investment with services needed by a dental practice
Targeted employee education programs to improve ethos of a business and employee’s confidence/performance
Collecting and displaying data in a manner that leads to increased understanding for all members of a practice or business
Increased awareness of the variables that may influence evaluation outcomes