EPE TIG Week: Johanna Morariu on an Evaluation Approach for an Environmental Think Tank & Advocacy Organization

Hi! I’m Johanna Morariu, Senior Associate with Innovation Network. Innovation Network (http://www.innonet.org/) is an evaluation consulting firm that provides evaluation consulting services to nonprofits and funders, and works to build the sector’s evaluation capacity. I lead Innovation Network’s environmental evaluation work, with a strong concentration on environmental advocacy evaluation.

A few months ago we began working with the Post Carbon Institute to develop an organizational evaluation framework. The Post Carbon Institute provides individuals, communities, businesses, and governments with the resources needed to understand and respond to the interrelated economic, energy, environmental, and equity crises that define the 21st century. As the organization identifies with being both a think tank organization and an advocacy organization, the evaluation team knew both elements needed to be present in the evaluation framework.

Since Innovation Network has been at the forefront of the advocacy evaluation field development, over the years we have amassed ample advocacy evaluation resources (free log in required). Early on we identified a need to supplement our existing expertise with a field scan of think tank evaluation literature. We found four resources that were particularly helpful, and we synthesized the resources to create a matrix of assessment areas.

Rad Resources:

• Donald E. Abelson (2010). Is Anybody Listening? Assessing the Influence of Think Tanks. Chapter 1 in the edited volume, Think Tanks and Public Policies in Latin America.
• Richard Bumgarner, Douglas Hattaway, Geoffery Lamb, James G. McGann, and Holly Wise (2006). Center for Global Development: Evaluation of Impact. Arabella Philanthropic Investment Advisors, LLC for the Bill & Melinda Gates Foundation, the William and Flora Hewlett Foundation, the John D. and Catherine T. MacArthur Foundation, and the Rockefeller Foundation.
• Ingie Hovland (2007). Making a Difference: M&E of Policy Research. Working paper 281 for the Overseas Development Institute, London, UK.
• James G. McGann (2006). Best Practices for Funding and Evaluating Think Tanks & Policy Research. McGann Associates for the William and Flora Hewlett Foundation.

The next step was to fine tune the areas of assessment for think tank evaluation and advocacy evaluation with interview data collected from key informants. Then, in consultation with key staff from the Post Carbon Institute, we created an organizational theory of change. The theory contains information about the organization’s mission; audiences; strategies; focusing events, crises, and windows of opportunity; desired shifts, and impact.

From the theory of change, the evaluators developed corresponding outcomes, indicators, and data collection tools, designed to provide actionable information about organizational strategy. A three tier approach to data collection was recommended—ongoing (e.g., meeting tracking, media tracking, champion tracking), annual (e.g., Bellwether interviews, partner survey, capacity assessment), and as needed (e.g., Intense Period Debrief).

The American Evaluation Association is celebrating Earthweek with our colleagues in the Environmental Program Evaluation AEA Topical Interest Group. The contributions all this week to aea365 come from our EPE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.