I’m Sue Hamann and have worked as an Evaluator for almost 40 years and am currently employed at the National Institutes of Health as a Health Scientist and Science Evaluation Officer. Many people new to evaluation are assigned to work as evaluators. My tips today are for those novice evaluators who need some help getting started with an evaluation plan.
Evaluators and many program staff are accustomed to summarizing programs or policies or other topics of evaluation with a graphic tool called a logic model. (I am going to just write “program evaluation” from now on, but it will mean any subject of an evaluation.) A logic model describes the stage of development of a program. The columns in a logic model are typically labeled as follows:
Standard Program Logic Model Framework
| Inputs | Activities | Outputs | Outcomes | Outcomes | Outcomes |
| (resources) | (planned program components) | (delivered and quantified program | Short-term | Intermediate | Long-term |
As an advocate for needs assessment as the starting point in program planning, I always modify the typical program logic model to show why a program is developed, as follows:
Comprehensive Program Logic Model Framework
| Antecedent conditions | Inputs | Activities | Outputs | Outcomes | Outcomes | Outcomes |
Evaluations must be tailored to the stage of the program. For example, you cannot evaluate performance unless the program has been implemented and you cannot evaluate outcomes until the program has been up and running. These stages used to be known as the Evaluation Hierarchy and have more recently been termed Evaluation Domains. The domains differ by evaluation questions, types, and methods.
Evaluation Domains Mapped onto Comprehensive Program Logic Model
|
Program Logic Model Framework
|
||||||
| Antecedent conditions | Inputs | Activities | Outputs | Outcomes | Outcomes | Outcomes |
|
Evaluation Logic Model Framework
|
||||||
| Needs Assessment | Evaluation of Program Design and Theory, Evaluability Assessment | Evaluation of Program Implementation and Processes,
Performance Monitoring |
Evaluation of Early and Intermediate Outcomes | Evaluation of Impact | Evaluation of program cost and efficiency | |
Lessons Learned:
- Adding Antecedent Conditions as the first column in your program logic model ensures that you understand why a program or policy was created and that you are focusing on relevant outcomes and impacts.
- The elements or columns in a logic model appear linear but are frequently iterative.
- The elements or columns in a logic model should be logically aligned.
- Developing the evaluation plan using a logic model framework can clarify your thinking and ensure that the evaluation is tailored to the stage of a program. Appropriate evaluation questions and methods differ depending on the stage of the program.
Rad Resources:
For Evaluation Planning see:
- Scriven M. (2015) Key Evaluation Checklist.
- Stufflebeam, DL. (2004) Evaluation plans and operations checklist.
- Research, Technology, and Development Topical Interest Group of the American Evaluation Association. (February 2015). Evaluating outcomes of publicly-funded research, technology and development programs: Recommendations for improving current practice.
- Rossi PH, Lipsey MW, Henry GT. (2018), Evaluation: A Systematic Approach, 8th Edition. Sage Publishing.
The American Evaluation Association is celebrating Needs Assessment (NA) TIG Week with our colleagues in the Needs Assessment Topical Interest Group. The contributions all this week to aea365 come from our NA TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.