My name is Helen Holmquist-Johnson and I teach a course in program evaluation to Master’s of Social Work students. This tip is on the facilitation of an evaluability assessment – a pre-evaluation tool.
Hot Tip: The evaluability assessment has been likened to grooming the slopes before skiing which is a fitting metaphor to use with my students at Colorado State University (Patton, Utilization-focused evaluation, 3rd ed.) We begin with the following two steps and then I provide more detailed information on how to go about conducting this important first step.
- Identify a program or part of a program for evaluation. Seek clarity regarding which part of the program will be evaluated.
- With program staff identify the primary purpose of the evaluation.
Start to collect the following:
- A brief program description (a program brochure or information available on the internet is a good starting place). When was the program initiated? (Is it a newer program or an established program?)
- What are the program components? What are the major clusters of activities carried out by the program? (e.g., case management, parenting classes, counseling, food distribution, pain management, etc.)
- A copy of the mission, goals, and objectives of the program or project. These may appear in the written materials about the program, such as a grant proposal for the program. What issues or problems are being addressed the program? If the program’s mission and goals are not explicit, you may be able to elicit how staff (or other stakeholders) operationalize these by asking the following questions (Patton):
- What are you trying to achieve with your clients?
- If you are successful, how will your clients be different after the program than they were before?
- What kinds of changes do you want to see in your clients?
- When your program works as you want it to, how do clients behave differently? What do they say differently? What would I see in them that would tell me they are different?
- A list of beginning evaluation questions that the agency is interested in answering.
- What information might be needed to answer the evaluation questions? Is it accessible? Available? How? From where?
- A list of the potential stakeholders, based upon the evaluation purpose and beginning evaluation questions.
These materials will help you appraise the stage of the program and the program’s readiness for further evaluation work. Additionally, you will take the next steps better prepared to recognize aspects of utilization and barriers that might be encountered along the way.
For more, check out this Rad Resource: http://www.jrsa.org/pubs/juv-justice/evaluability-assessment.pdf
The American Evaluation Association is celebrating this week with our colleagues at the Southern California Evaluation Association (SCEA), an AEA affiliate. The contributions all this week to aea365 come from SCEA members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.