We are Ehren Reed and Johanna Morariu, Senior Associates of Innovation Network. We work with foundations and nonprofits to evaluate and learn from programs, projects, and advocacy endeavors. For more than fifteen years, Innovation Network has been an intermediary in the philanthropic and nonprofit sectors—our mission is to build the evaluation capacity of people and organizations.
For some time, the evaluation field has lacked up-to-date, sector-wide data about nonprofit evaluation practice and capacity. We thought that such information would not only be helpful to us as evaluation practitioners, but could also inform a wide variety of other audiences, including nonprofits, funders, and academics. The State of Evaluation project (www.stateofevaluation.org) is Innovation Network’s answer to this need. In May 2010 we launched a survey to a nationally representative sample of 36,098 nonprofits (all were 501(c)3 organizations) obtained from GuideStar. We received 1,072 complete responses from representatives of nonprofit organizations (for a response rate of 2.97%). Survey results are generalizable to all U.S.-based nonprofits, with a margin of error of plus or minus 4%.
Lessons Learned:
With a tip of the hat to David Letterman, here are the “Top Ten” highlights from State of Evaluation 2010: Evaluation Practice and Capacity in the Nonprofit Sector:
1. 85% of organizations have evaluated some part of their work in the past year.
2. Professional evaluators are responsible for evaluation in 21% of organizations. (For more than half of nonprofit organizations, evaluation is the responsibility of the organization’s leadership or board.)
3. 73% of organizations that have worked with an external evaluator rated the experience as excellent or good.
4. Last year, 1 in 8 organizations spent no money on evaluation. (Less than a quarter of organizations devoted the minimum recommended amount of 5% of their budget to evaluation.)
5. Half of organizations reported having a logic model or theory of change, and more than a third of organizations created or revised the document within the past year.
6. Quantitative evaluation practices are used more often than qualitative practices.
7. Funders were named the highest priority audience for evaluation.
8. Limited staff time, limited staff expertise, and insufficient financial resources are barriers to evaluation across the sector.
9. Evaluation was ranked #9 of a list of ten organizational priorities. Fundraising was #1, and research was #10.
10. 36% of nonprofit respondents reported that none of their funders supported their evaluation work. (Philanthropy and government sources are most likely to fund nonprofit evaluations.)
This report—State of Evaluation 2010—marks the first installment of this project. In two years, we will conduct another nationwide survey and update our findings. To learn more about the project, please visit www.stateofevaluation.org.
This contribution is from the aea365 Tip-a-Day Alerts, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org. Want to learn more from Ehren and Johanna? They’ll be presenting as part of the Evaluation 2010 Conference Program, November 10-13 in San Antonio, Texas.
Hi Rebekah, sorry for the delay in response, and I hope you are still listening in! If you do a literature review of recommendations for evaluation budgeting, nearly all sources talk about budget as a percentage of *program budget* (vs. organizational budget). Those sources (CDC, Kellogg, etc.) recommend on average that 5 – 10% of program budget be spent on evaluation. (We cite a few sources in the full length report, http://www.stateofevaluation.org)
In Innovation Network’s work, we like to push for evaluation and strategy to be intimately connected–two sides of the same coin. We want organizations to be learning organizations, which often means breaking out of the program silos that are all too common.
When it comes to budgeting guidance for the time, training, and tools an organization needs to be a learning organization…well, I’m not sure that is guidance that can be generalized into an easy rule of thumb such as percentage of budget.
All of this is a long way of saying that there isn’t a transitive relationship between the percentage of budget spent on evaluation at the program level and the percentage of budget spent on evaluation at the organizational level. It’s apples to oranges.
My opinion is that if only 23% of organizations are spending the recommended 5% or more of their organizational budgets on evaluation (which took into account staff time, tools, etc.), then the vast majority of organizations are not investing in being learning organizations. And if they’re not investing in being learning organizations, they’re not nearly as strategic and effective as they could be. Hope that helps!
-J
For those interested, the materials Johanna Morariu used at her presentation on the State of Evaluation Practice and Capacity in the Nonprofit Sector in Session 883 at the Evaluation 2010 Conference can be accessed here: http://bit.ly/MorariuEval2010.
I shared this blog entry with my institutional leadership and I have a request. Can you share a little background about the recommended 5% budget line for evaluation for institutions? Is that an Innovation Network recommendation for non-profits as a way to “make sure” evaluation gets considered and used in all areas? I am hoping to use your response to justify my institution making this change, so any references/ sources you can provide would be helpful. thank you