Hello! We are Katrina Brewsaugh, and Stephen Brehm. We are the Data Team at One Hope United, an agency providing a wide range of child and family services in Illinois, Missouri, and Florida.
Evaluators are often tasked with measuring how a program is meeting performance measures and making recommendations on whether programs that are not meeting targets can make improvements. When large gaps exist between established targets and actual performance, programs may be under pressure to close the gap in a very short amount of time, often without additional supports. Rational Target Setting Methodology (RTSM; Zirps, 2003) is an approach that uses past performance and an assessment of the current environment to set an ambitious, yet realistic trajectory for improvement or to attain desired results.
Hot Tip: RTSM can be used by evaluators as a framework to facilitate discussions with internal and external stakeholders about what amount of improvement can reasonably be expected in a 12-month time span and what resources would be needed to make a specific level of improvement.
Hot Tip: RTSM takes into account the level of support in policy, priority, resources, and training with each area weighted from zero to three. Low weighting in one area can be offset by high weighting in another. For instance, low resource allocation may be offset by a strong emphasis on staff training. Evaluators can use this knowledge to help programs design their own action plans for improvement, while potentially increasing their buy-in to the change effort.
Hot Tip: The sum of the quadrant ratings corresponds to a range in anticipated improvements. Closing large gaps requires a multi-year effort; even if all quadrants were given the maximum rating of 3, there is no way to make more than a 50-point improvement in one year. It is important to remember that our progress toward our goals will be proportional to the resources and efforts we direct.
Lesson Learned: It is vital that discussions and ratings are realistic. In our practice with RTSM, groups have at times been overly optimistic about one or more categories and are then disappointed when the amount of change is less than expected. Should any of the categories go through significant change during the year (e.g. unexpected funding cuts) the weights would have to change accordingly as well.
Rad Resource: For more information on RTSM, please visit the AEA eLibrary where we have posted materials from our 2011 Conference session or contact Katrina Brewsaugh at kbrewsaugh@onehopeunited.org.
Rad Resource:
Zirps, F. (2003). Still doing it right: A guide to quality for human service agencies. Albuquerque, NM:IQAA Books.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Katrina and Stephen, how can one get a hold of the resource that you mention, the Zirps book? I have been unable to find reference to it online. Alternatively, is there another source for learning more about the theory or research behind the Rational Target Setting Methodology? Thank you for sharing, I’d like to learn more.
Readers – be sure to click through and see the handouts in the eLibrary, really helped me to understand the concepts explained here. Thank you both so much for sharing.