Ed Eval TIG Week: The Program Evaluation Standards – Not Just for the Bookshelf… by Thanos Patelis and Emily Dickinson

Greetings from Thanos Patelis and Emily Dickinson! We are researchers at the Human Resources Research Organization (HumRRO), a non-profit research organization. We are happy to talk about a favorite Rad Resource of ours—The Program Evaluation Standards.

Every time we undertake a new project, we are reminded of the multiple demands that come with an evaluation. Not only do the scientific demands of a rigorous design befall the evaluator, but also the demands to make the evaluation efficient, timely, and responsive to the needs of clients.

To ensure the quality of our evaluation efforts, we have found it helpful to ground our work in the Program Evaluation Standards (heretofore referred to as The Standards). The Standards are organized around five components: Utility (ensuring stakeholders find the evaluation processes and products valuable in meeting their needs), Feasibility (ensuring effectiveness and efficiency), Propriety (ensuring proper, fair, legal, right and just evaluations), Accuracy (ensuring the dependability and truthfulness of the evaluations), and Evaluation Accountability (ensuring adequate documentation and improvement of evaluations). Therefore, The Standards provide an excellent organizing framework, encompassing all the requirements of a high-quality evaluation design.

Today’s Hot Tips focus on the Utility component of The Standards. A high-quality evaluation is one that meets the needs of stakeholders, and so the utility of processes and products should be considered during even the earliest stages of design planning.

Hot Tip #1: Involve stakeholders early and often. Standard U3 indicates that the purposes of an evaluation should be established based on the needs of stakeholders and revised as needed through the lifecycle of the evaluation project. Start by clearly stating the goals of the project in terms that your key stakeholders agree upon, then keep the lines of communication open so that adjustments can quickly be made in response to changing circumstances or priorities.

Hot Tip #2: Keep the conversation going. Standard U7 further reinforces the idea that continuous communication is key by indicating that we should routinely monitor and address the information needs of stakeholders. Sharing preliminary results could lead stakeholders to adjust near term goals or objectives, for example. Similarly, unanticipated changes in the program or in the environment in which it is being carried out could have implications not only for the goals of the evaluation, but for the way data are collected and interpreted. But you won’t know if you aren’t listening!

In closing, if you haven’t already, dust off that copy of The Standards and see where it takes you!

Rad Resource: The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.)

The American Evaluation Association is celebrating Ed Eval TIG Week with our colleagues in the PreK-12 Educational Evaluation Topical Interest Group. The contributions all this week to aea365 come from our Ed Eval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.