{"id":20647,"date":"2019-05-13T03:39:47","date_gmt":"2019-05-13T07:39:47","guid":{"rendered":"https:\/\/aea365.org\/blog\/?p=20647"},"modified":"2019-05-11T10:53:35","modified_gmt":"2019-05-11T14:53:35","slug":"sim-tig-week-moving-beyond-scale-the-importance-of-capturing-feedback-from-your-target-audience-by-sachi-shenoy","status":"publish","type":"post","link":"https:\/\/aea365.org\/blog\/sim-tig-week-moving-beyond-scale-the-importance-of-capturing-feedback-from-your-target-audience-by-sachi-shenoy\/","title":{"rendered":"SIM TIG Week: Moving beyond scale: the importance of capturing feedback from your target audience by Sachi Shenoy"},"content":{"rendered":"

I am Sachi Shenoy<\/strong>, the Co-Founder and Chief Impact Officer of Upaya Social Ventures<\/a>. Upaya\u2019s mission is to create dignified jobs for the poorest of the poor by investing in and consulting with small business entrepreneurs anchored in marginalized communities. To date, our 16 investee companies have cumulatively created more than 12,000 jobs.<\/p>\n

In the decade-plus I\u2019ve been doing this work, I\u2019ve often heard mission-driven organizations talk about the importance of scale. While reporting large and wide outreach is a commendable goal, it should not preclude measuring the depth<\/em> of impact. It\u2019s important to ask how a program benefited its target audience in the intended way. If we have created new jobs, for example, have jobholders\u2019 incomes increased, especially compared to their old jobs?<\/p>\n

Hot Tip:<\/strong><\/p>\n

Mission-driven organizations should consider surveying a small sample of their beneficiaries. With each company Upaya invests in, we send third-party enumerators to visit a random sample of jobholders for a 10-minute survey. We try to learn how the program benefited its target audience and what program modifications could optimize effectiveness.<\/p>\n

We spend on average $100,000 annually, or 10 percent of our budget, on these activities. We have always believed the benefits outweigh the costs, and that the same holds true for other organizations. Creative methods\u2014such as employing student interns, leveraging free mobile survey tools like KoBoToolbox<\/a>, and pulling it all together with systems like Impact Cloud by SoPact<\/a>\u2014can make this process straightforward and cost-effective.<\/p>\n

Lessons Learned:<\/strong><\/p>\n

The nearly 1,200 surveys<\/a> we have collected over the years have provided invaluable insights. For example, one investee company found jobholders reported a 200 percent increase in their household incomes and were spending considerably more on food and consumer goods. But one year into these jobs, despite the uptick in income, there was little to no movement on longer-term indicators like improvements in housing condition. Midline surveys revealed jobholders had no safe way to save\u2014they were not getting approved for bank accounts, and the proverbial \u201cstuffing the mattress\u201d method of savings was prone to theft. It was no wonder then, that short-term indicators were highly positive, but longer-term indicators were flat. This prompted our entrepreneur to devise savings schemes, such as automatic payroll deductions, for his employees. Early pilot testing showed good promise, and jobholders viewed this as an additional benefit of staying in the job.<\/p>\n

For any organization, surveys can unearth valuable customer insights that can improve program effectiveness. The purpose of this information is not to \u201cprove impact;\u201d other more rigorous, resource-intensive methods like randomized control trials (RCTs) can establish causal links between program activities and outcomes. We disagree, however, with the binary notion that one must either invest in RCTs or do nothing at all. We believe there is a middle ground\u2014one that involves gathering feedback from our target audience, in their own words, and constantly refining our programs to best serve their needs.<\/p>\n

The American Evaluation Association is celebrating Social Impact Measurement Week <\/strong>with our colleagues in the Social Impact Measurement Topical Interest Group<\/strong>. The contributions all this week to aea365 come from our SIM TIG<\/strong> members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the<\/em> aea365 webpage<\/em><\/a> so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to <\/em>aea365@eval.org<\/em><\/a>. aea365 is sponsored by the<\/em> American Evaluation Association<\/em><\/a> and provides a Tip-a-Day by and for evaluators.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

I am Sachi Shenoy, the Co-Founder and Chief Impact Officer of Upaya Social Ventures. Upaya\u2019s mission is to create dignified jobs for the poorest of the poor by investing in and consulting with small business entrepreneurs anchored in marginalized communities. To date, our 16 investee companies have cumulatively created more than 12,000 jobs. In the […]<\/p>\n","protected":false},"author":193,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"site-sidebar-layout":"default","site-content-layout":"default","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"default","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[1479],"tags":[],"class_list":["post-20647","post","type-post","status-publish","format-standard","hentry","category-social-finance"],"_links":{"self":[{"href":"https:\/\/aea365.org\/blog\/wp-json\/wp\/v2\/posts\/20647","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/aea365.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/aea365.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/aea365.org\/blog\/wp-json\/wp\/v2\/users\/193"}],"replies":[{"embeddable":true,"href":"https:\/\/aea365.org\/blog\/wp-json\/wp\/v2\/comments?post=20647"}],"version-history":[{"count":1,"href":"https:\/\/aea365.org\/blog\/wp-json\/wp\/v2\/posts\/20647\/revisions"}],"predecessor-version":[{"id":20648,"href":"https:\/\/aea365.org\/blog\/wp-json\/wp\/v2\/posts\/20647\/revisions\/20648"}],"wp:attachment":[{"href":"https:\/\/aea365.org\/blog\/wp-json\/wp\/v2\/media?parent=20647"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/aea365.org\/blog\/wp-json\/wp\/v2\/categories?post=20647"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/aea365.org\/blog\/wp-json\/wp\/v2\/tags?post=20647"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}