AEA365 | A Tip-a-Day by and for Evaluators

CAT | Social Impact Measurement

This is Karim Harji and Ted Jackson from ET Jackson & Associates, and our contribution on social impact measurement (SIM) focuses on training and curriculum development for evaluators and practitioners.

Lessons Learned:

Training and Cross-Sector Collaboration

Impact investing – a field that has seen much growth over the last decade, through a combination of strategic grant making and industry development – is steadily gaining attention and traction.. While this is new terrain for evaluators, there are several resources on both impact investing and impact measurement, and strategies for evaluators, for engaging in this area.

In collaboration with several African institutions – the CLEAR Centre at Wits University and Greater Capital in South Africa; and the Venture Capital Trust Fund, GIMPA Centre for Impact Investing, and the Institute for Policy Alternatives in Ghana – together with funding from the Rockefeller Foundation and IDRC, we developed and delivered executive education training workshops in Ghana and South Africa.

These workshops not only brought evaluators, practitioners and policymakers, but also stimulated cross-sector conversation on the relevance of measurement. Participants shared approaches and tools for their local contexts, and emphasized the need for greater cross-sector collaboration among investors, fund managers, social enterprises, evaluators and governments.

Developing Curriculum and Resources

Inspired by these sessions and the range of perspectives that were represented, we designed an open source curriculum on “Evaluating Impact Investing”. The course is built around three broad themes—building the field of impact investing, measuring the success of impact investments, and understanding the special issues emanating from the African context.

Impact investing in Africa is characterized by much dynamism, and as such, the course consists of 24 different modules – on topics such as Evaluation Approaches, Standards, Global Goals, Household Impacts, Negative Outcomes, and Evaluation Costs – with supporting notes, examples, case profiles, exercises and readings. We explore the use of well-known approaches such as Theory of Change, as well as contextual adaptations for engaging with the private sector.

This curriculum is a resource that we hope that academic organizations and evaluation networks can use and adapt for their own needs. That said, there is much more to be done, including more case studies authored by African researchers, and evaluation tools that are adapted for use within each region. We welcome contributions and suggestions from evaluators, as well as those who are designing and delivering training and education in evaluation.

(click for larger image)

friday-graphic-harji-jackson

Interested in learning more? For more information about the SIM TIG, see here. To join the SIM TIG, see here.

The American Evaluation Association is celebrating Social Impact Measurement Week with our colleagues in the Social Impact Measurement Topical Interest Group. The contributions all this week to aea365 come from our SIM TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

 

 

No tags

My name is Kelly McCarthy and I am the Director of Impact Measurement & Management at the Global Impact Investing Network – or GIIN – a nonprofit whose purpose is to increase the scale and effectiveness of impact investing.  While we don’t make investments ourselves, we work alongside those who do, along with many other actors seeking to advance impact investing practice and channel more capital to generate positive social and environmental outcomes.

Impact measurement and management (IMM) is core to effective impact investing

Though I am a new entrant to the AEA community, I share the same aim as many others in this forum: To collaborate with those who strive to advance sound IMM and evaluative practices to drive more effective decision-making. In our case, decision-making anchors on using social and environmental data to inform the allocation of investment capital and optimize the potential for it to deliver positive social and environmental results. One way we do this is through our oversight of the publicly available IRIS catalog of social and environmental performance metrics developed with input from evaluation professionals and investment practitioners. Other such ways include working with funds around the world to deepen the IMM  capacities of their teams and supporting the growing profession of impact analysts in the financial sector.

To further this aim, as other contributors this week have highlighted, one important focus area for 2017 will be deepening meaningful connection (and action) points between the impact investment community and evaluators and other IMM stakeholders.

Action 1: Translating Existing IMM Practices for Investors

Impact measurement and management often engages diverse implementing stakeholders from a variety of disciplines, including impact analysts, evaluators, and impact investment strategists and decision-makers. As a result a large number and diversity of industry and proprietary IMM approaches (frameworks, ratings, and methodologies) has emerged. From the perspective of many impact investors, this creates confusion, is viewed as lack of consistency in the market, and presents a barrier for effective use of IMM in general.

As a very first step to address this challenge, we are developing a publicly available “map” of existing credible initiatives, methodologies, and assessment tools which are currently in use by – or can be translated to – the investment context  (e.g., due diligence, metrics selection, etc.). We welcome you to join the GIIN in these initial efforts to deepen understanding of the appropriateness of different methods for impact investing.

(click for larger image)thursday-chart_how-social-enviro-performance-is-measured_giin-2016-survey

If you or someone you know is interested in contributing to this effort, please reach out to kmccarthy@thegiin.org

Rad Resources:

 

For more information about AEA’s SIM TIG, see here. To join the SIM TIG, see here.

The American Evaluation Association is celebrating Social Impact Measurement Week with our colleagues in the Social Impact Measurement Topical Interest Group. The contributions all this week to aea365 come from our SIM TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Hi, my name is Sara Olsen and I’m a founding board member of Social Value US and the CEO of SVT Group, an impact management services firm I founded to “make impact management ubiquitous worldwide.” And I’m Kate Ruff.  I am an accounting prof at Carleton University in Ottawa, Canada. I study how organizations measure and report their social results (outputs, outcomes impact), focused on accounting problems, such as standard setting, materiality, relevance, presentation and valuation, as they apply to social impact.

Lessons Learned: There has been a lot of talk in impact investing and other circles about the need for “impact measurement standards,” but many if not most efforts to promote a fixed set of measures have met with overt or covert resistance, and failed to gain traction.

Recently we co-authored an article discussing a potential solution to this conundrum. Standardized indicators of impact can only get us so far because they are often insufficiently relevant to the decisions these different parties must make about how to understand and improve their impact. At the same time, having complete discretion over what to measure makes measures incomparable and hinders the development of an efficient social capital marketplace since it becomes virtually impossible to understand or improve impact at the portfolio, sector or systemic level. What’s an impact investor to do?

Drawing lessons from the evolution of financial accounting, we propose that the solution is skilled impact analysts:

“[I]t is possible to achieve comparability by focusing on the analytical skills needed to compare social impacts without mandating a rigid set of required metrics. The premise is that efficient capital markets demand analysts who are capable of interpreting and comparing apples and oranges. Why? Because they understand fruit. The market is best served when each organization can measure its social impact in the way that is most meaningful and insightful to its aim and operations, as long as it follows common principles for good measurement.”

We term this approach “bounded flexibility,” or choice within limits. We assert that it, “coupled with consistency (pick a method and stick to it) and disclosure (fine print gives details on methods used), is the solution to understanding impact that will enable the social capital markets to flourish.

Rad Resources:

  • For more information about the SIM TIG, see here. To join the SIM TIG, see here.
  • Social Value International is a professional home where folks from different domains (accounting, finance, philanthropy, evaluation, sociology, environmental economics, data science etc) have come together to work on changing the way society accounts for value. SVI has chapters in the US (newly), Canada and 18 other countries worldwide and members in many more.
  • Our article in the Stanford Social Innovation Review delves deeper into this issue and what it looks like in practice. It develops ideas that were first published in Ruff, K “The Role of Intermediaries in Social Accounting: Insights from effective transparency systems” in Mook, L. (Ed.) Accounting for Social Value. University of Toronto Press, 2013. 

The American Evaluation Association is celebrating Social Impact Measurement Week with our colleagues in the Social Impact Measurement Topical Interest Group. The contributions all this week to aea365 come from our SIM TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Hi, my name is Donna Loveridge and I work with the Donor Committee for Enterprise Development (DCED), a forum for learning about the most effective ways to create economic opportunities for the poor, in line with the SDGs – based on practical experience in private sector development.

International development organisations, such as DCED members, have traditionally used grants to create economic opportunities for the poor. Now they are increasingly adding impact investing to their repertoire of strategies. But how do impact investors know social and environmental impacts are occurring? Recently, I compared current practices in social impact measurement (SIM) in the impact investing field and results measurement (or M&E) in international development to find the similarities and differences (the full paper can be found here).

Lessons Learned: Here are some insights:

  1. Impact: In international development, the term ‘impact’ has traditionally meant long-term changes (positive, negative, intended, unintended) produced by an intervention directly and indirectly. In impact investing, impact can mean short, medium or long-term changes, including outputs and changes that a business may make to the way it operates, as a means to an end, e.g. more community or employee consultation.
  2. Standardisation: Impact investing is more interested in standardising results measurement approaches and methods to compare investment opportunities and returns than international development. This can be challenging since appropriate approaches and methods are those that consider factors like the investment’s goals; implementation model; location; timescales; and stakeholders such as investor, investee and beneficiaries.
  3. Intermediaries: Currently, impact investing and international development place few expectations on businesses to measure their social and environmental impact – greater onus is placed on intermediaries like fund or programme managers.
  4. Monetary value: Impact investing places greater priority on assessing the monetary value of social impacts than most development programmes, although there has been a strong push towards assessing the value for money of development programmes in recent years by DFID, including using approaches such as social return on investment.

Lessons learned:

  • Communicate –Take the time to understand what people from the other field mean when they use certain terms or words.
  • Look to see what is already out there – exchanging views, sharing resources and cross-fertilisation will contribute to the development of SIM and results measurement.

Rad Resources:

More than 100 private sector development programmes use the DCED Standard for results measurement. Check out practical guidelines for results measurement on challenge funds, a type of development programme with some similar attributes to some impact investments, here.

In 2017, the DCED is looking at what information investees value enough to measure and evaluate themselves; and how results (financial, social and environmental) are attributed to different investors. Subscribe to the DCED newsletter here to keep up with developments.

Interested in learning more? For more information about the SIM TIG, see here. To join the SIM TIG, see here.

The American Evaluation Association is celebrating Social Impact Measurement Week with our colleagues in the Social Impact Measurement Topical Interest Group. The contributions all this week to aea365 come from our SIM TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Hi.  This is Jane ReismanFounder of ORS Impact. I currently describe myself as a social impact advisor as I aspire to create connections between what we do as evaluators and the impact that we hope to achieve.

I’ve been actively engaged- along with other AEA members — in building bridges between the evaluation community and the impact investing community which focuses investments on social/environmental impact. Our efforts are paying off.   Two exciting developments in December 2016 signal progress in bridge-building.   Several AEA members were invited to participate in key events led by the impact investing community:  1) the Global Impact Forum’s strand on measurement and evaluation hosted by the Global Impact Investing Network and 2) the World Economic Forum’s workshop “Accelerating Impact Measurement and Management.”

Evaluators contributed fundamental perspectives during these events that are influencing how the impact investing community shapes the next generation of approaches to impact measurement and evaluation.

Some evaluative perspectives squarely align with existing thinking and practices among impact investors; others push the boundaries.

  • Beneficiary voice and stakeholder engagement are essential to defining and determining value.
  • Diverse approaches utilized by evaluators can be blended with existing and emerging impact measurement approaches to create new and robust methods.
  • Impact measurement and evaluation produce critical data for guiding decisions and strategic thinking.
  • Outputs alone are not sufficient proxies for assessing social and environmental changes.
  • Complexity thinking is relevant to designing effective solutions for social and environmental problems.
  • Theory of change is valuable for clarifying impact and measurement priorities.
  • Building an evidence base for impact investing and other market solutions is necessary for strategic and effective investments.
  • Specific skill sets are required for successfully executing in measurement and evaluation.

AEA’s Social Impact Measurement TIG launched in 2016 will provide opportunities to share and hone our thinking, practice, and evidence in this emergent area.

Hot Tips:

  1. Tune into the momentum for developing the next generation of impact measurement and evaluation and seek opportunities to contribute to these efforts
  2. Recognize that the impact investing community is receptive to contributions from allied fields like evaluation
  3. Respect that impact investing has its own culture and mindset about measurement and evaluation and of our perspectives may push thinking in new directions.
  4. For more information about the SIM TIG, see here. To join the SIM TIG, see here.

Rad Resource:  Situating the Next Generation of Impact Measurement and Evaluation for Impact Investing by Jane Reisman and Veronica Olazabal.

The American Evaluation Association is celebrating Social Impact Measurement Week with our colleagues in the Social Impact Measurement Topical Interest Group. The contributions all this week to aea365 come from our SIM TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

This is Veronica Olazabal and Shawna Hoffman of The Rockefeller Foundation’s Evaluation team. We have the pleasure of introducing this week’s focus on social impact measurement (SIM) for innovative finance and market solutions.

Hot Tips: What does innovative finance and market solutions have to do with evaluation, you ask?

A lot! In recent years, new forms of capital have been mobilized at unprecedented rates to create social good. The responsibility to contribute to social and environmental outcomes no longer falls solely on traditional government, philanthropic, and non-governmental actors; innovative market players, such as impact investors, are beginning to play an increasingly important part as well. This is significant because it exponentially increases the pool of funds available for social impact (see figure below). The emergence of these new actors presents great opportunity for the social sector, but also carries implications for how ‘we’ think, talk about, and—of course—evaluate impact.sunday-post-image-as-a-jpeg

How can evaluation work with innovative finance and market solutions?

At last year’s AEA conference and pre-conference Impact Convergence our colleagues asked a lot of important questions about how to bridge the evaluation and impact investing and market solutions spaces. Below are the top five questions that evaluators are asking about measurement and evaluation of innovative finance and market solutions:

  • How can we evolve or adapt our practices, goals, tools, and language to enable us to effectively engage with market actors?
  • What are the data needs of market actors? How can differing considerations such as cost, timeliness and risk be taken into account?
  • To what extent is attribution important? Do private sector players care about being able to attribute change in the same way that traditional development players often do?
  • How can evaluators and investors learn from each other and leverage one another’s respective strengths, including the private sector’s strengths vis-à-vis robust accounting, efficiency, and effective use of technology, and evaluation’s adherence to principles of objectivity, rigor and ethics?
  • How might evaluators frame their value proposition to investors and other market actors?

To help respond to these questions, this week we will hear from leading thinkers in this space on how they have responded to this new demand for assessing impact.

Rad Resources: Interested in learning more?

  • For more information about the SIM TIG, see here. To join the SIM TIG, see here.
  • Check out this reading list on Evaluating Impact Investing
  • Follow along this weeks’ AEA365 Tip-of-the-Day and hear from some of the leading voices on this topic.

The American Evaluation Association is celebrating Social Impact Measurement Week with our colleagues in the Social Impact Measurement Topical Interest Group. The contributions all this week to aea365 come from our SIM TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

No tags

Archives

To top