Ed Eval TIG Week: The Program Evaluation Standards – Not Just for the Bookshelf… by Thanos Patelis and Emily Dickinson

Greetings from Thanos Patelis and Emily Dickinson! We are researchers at the Human Resources Research Organization (HumRRO), a non-profit research organization. We are happy to talk about a favorite Rad Resource of ours—The Program Evaluation Standards.

Every time we undertake a new project, we are reminded of the multiple demands that come with an evaluation. Not only do the scientific demands of a rigorous design befall the evaluator, but also the demands to make the evaluation efficient, timely, and responsive to the needs of clients.

To ensure the quality of our evaluation efforts, we have found it helpful to ground our work in the Program Evaluation Standards (heretofore referred to as The Standards). The Standards are organized around five components: Utility (ensuring stakeholders find the evaluation processes and products valuable in meeting their needs), Feasibility (ensuring effectiveness and efficiency), Propriety (ensuring proper, fair, legal, right and just evaluations), Accuracy (ensuring the dependability and truthfulness of the evaluations), and Evaluation Accountability (ensuring adequate documentation and improvement of evaluations). Therefore, The Standards provide an excellent organizing framework, encompassing all the requirements of a high-quality evaluation design.

Today’s Hot Tips focus on the Utility component of The Standards. A high-quality evaluation is one that meets the needs of stakeholders, and so the utility of processes and products should be considered during even the earliest stages of design planning.

Hot Tip #1: Involve stakeholders early and often. Standard U3 indicates that the purposes of an evaluation should be established based on the needs of stakeholders and revised as needed through the lifecycle of the evaluation project. Start by clearly stating the goals of the project in terms that your key stakeholders agree upon, then keep the lines of communication open so that adjustments can quickly be made in response to changing circumstances or priorities.

Hot Tip #2: Keep the conversation going. Standard U7 further reinforces the idea that continuous communication is key by indicating that we should routinely monitor and address the information needs of stakeholders. Sharing preliminary results could lead stakeholders to adjust near term goals or objectives, for example. Similarly, unanticipated changes in the program or in the environment in which it is being carried out could have implications not only for the goals of the evaluation, but for the way data are collected and interpreted. But you won’t know if you aren’t listening!

In closing, if you haven’t already, dust off that copy of The Standards and see where it takes you!

Rad Resource: The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.)

The American Evaluation Association is celebrating Ed Eval TIG Week with our colleagues in the PreK-12 Educational Evaluation Topical Interest Group. The contributions all this week to aea365 come from our Ed Eval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

1 thought on “Ed Eval TIG Week: The Program Evaluation Standards – Not Just for the Bookshelf… by Thanos Patelis and Emily Dickinson”

  1. Dear Thanos and Emily,
    Thank you so much for your post regarding The Standards. In evaluation design there are so many aspects to consider. Using The Program Evaluation Standards to ensure that a high quality evaluation is developed is the key to success. I am new to evaluation and am just learning about the process of program evaluation design and the standards involved. I would imagine that there are instances when the different aspects of the components of design would conflict with one another. In the introduction to The Standards it discusses the idea that it is difficult to maximize quality in the aspect of each component. “In implementing the program evaluation standards, stakeholders must decide how to create the best quality evaluations based on prioritized needs.” (Yarbrough, 2011, p. xxii) I wondered if in your practice you have any advice to discovering and meeting these prioritized needs. You focus on the Utility component for your hot tip in this particular post and give some excellent suggestions. I wondered if you followed some sort of process in your practice when approaching The Standards and a new evaluation design. Do you feel that one of the components is more essential or even easier to being with than another? What would be your suggestion for someone who is just starting out? Thank you so much for your tips and your helpful post. I look forward to hearing back from you.
    Sincerely,
    Angie

    Reference
    Yarbrough, D. B. (2011). The program evaluation standards: A guide for evaluators and evaluation users. Retrieved August 5, 2019, from https://us.sagepub.com/sites/default/files/upm-binaries/35465_Intro.pdf

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.