IE TIG Week: Laura Beals and Rachel Albert on TIERS: A Tool for Allocating Evaluation Resources at Nonprofit Agencies

Hello evaluation colleagues! We’re Rachel Albert, Vice President of Learning and Impact, and Laura Beals, Director of Evaluation, from Jewish Family and Children’s Service (Boston, MA). Our department is responsible for the internal evaluation of 44 programs, collectively serving over 17,000 people a year.

At JF&CS, we face two challenges in evaluation management. First, we have multiple external and internal stakeholders – including foundations, federal and state grantors, individual donors, agency leadership, program managers, and staff – each of whom has a different perspective on what sorts of data they need. Second, instead of grants dictating the evaluation resources available to each program, our department is funded by overhead. This means it’s up to us to apportion our department’s evaluation resources thoughtfully across all 44 programs for maximum benefit.

Lessons Learned: To meet this challenge, we developed a tool we call TIERS (“Tool for Intra-agency Evaluation Resource Sharing”). TIERS helps us leverage our resources on each program to answer the questions most relevant to its stakeholders.

As you go higher in the pyramid, you are looking for stronger and stronger evidence that your program is achieving its intended impact. The pyramid is intended to be both cumulative and sequential: a program should not go up a tier until it has a robust implementation of the previous tier in place.

 Hot Tips:

• This is not a race: It’s ok to stop at whatever the right tier is for a given program based on its evaluation needs and staff resources.
• Higher tiers require more resources from both the internal evaluator and program staff.
• Do not underestimate the difficulty of establishing even just a rigorous Tier 1 across a large agency!

We presented this tool in a demonstration session at Eval 14; check out the AEA e-library for our slides and handout.

Rad Resources: If you are looking for additional information about resource allocation for evaluation, here are a few places to start:

• This TIERS framework was inspired by the evaluation theories and frameworks of others, including Michael Q. Patton’s Mountain of Accountability and Idealware’s “The Reality of Measuring Human Service Programs: Results of a Survey.”
• The Innovation Network’s “State of Evaluation 2012: Evaluation practice and capacity in the nonprofit sector,” which includes their recommendation that “organizations should budget from 5% to 10% of the organization budget for evaluation costs” (p. 5).
• State of the Nonprofit Sector Survey, by the Nonprofit Finance Fund, which includes data points on how often funders cover the costs of impact measurement (hint: 71% said “rarely” or “never”).
• A recent Markets for Good article by Laura Quinn on the cost of collecting data in nonprofit settings entitled “Forcing Nonprofits To Lie About Data.”

The American Evaluation Association is celebrating Internal Evaluation (IE) Topical Interest Group Week. The contributions all this week to aea365 come from our IE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.