Evaluators are often tasked with measuring how a program is meeting performance measures and making recommendations on whether programs that are not meeting targets can make improvements. When large gaps exist between established targets and actual performance, programs may be under pressure to close the gap in a very short amount of time, often without additional supports. Rational Target Setting Methodology (RTSM; Zirps, 2003) is an approach that uses past performance and an assessment of the current environment to set an ambitious, yet realistic trajectory for improvement or to attain desired results.
Hot Tip: RTSM can be used by evaluators as a framework to facilitate discussions with internal and external stakeholders about what amount of improvement can reasonably be expected in a 12-month time span and what resources would be needed to make a specific level of improvement.
Hot Tip: RTSM takes into account the level of support in policy, priority, resources, and training with each area weighted from zero to three. Low weighting in one area can be offset by high weighting in another. For instance, low resource allocation may be offset by a strong emphasis on staff training. Evaluators can use this knowledge to help programs design their own action plans for improvement, while potentially increasing their buy-in to the change effort.
Hot Tip: The sum of the quadrant ratings corresponds to a range in anticipated improvements. Closing large gaps requires a multi-year effort; even if all quadrants were given the maximum rating of 3, there is no way to make more than a 50-point improvement in one year. It is important to remember that our progress toward our goals will be proportional to the resources and efforts we direct.
Lesson Learned: It is vital that discussions and ratings are realistic. In our practice with RTSM, groups have at times been overly optimistic about one or more categories and are then disappointed when the amount of change is less than expected. Should any of the categories go through significant change during the year (e.g. unexpected funding cuts) the weights would have to change accordingly as well.
Zirps, F. (2003). Still doing it right: A guide to quality for human service agencies. Albuquerque, NM:IQAA Books.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to email@example.com. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.