I am Dawn Henderson, an Assistant Professor at Winston-Salem State University and former AEA GEDI (Graduate Education Diversity Initiative) intern. My experience as a GEDI afforded the opportunity to increase my knowledge in thinking about a program’s theory in evaluation. I used this in designing research and work with a school-based intervention. I want to share with you some tips I have used to help programs think about their particular theory and ways to evaluate and measure it. This is primarily targeted towards youth, so you can modify these tips for your specific needs.
Hot Tip: Conduct interviews with program staff and collect program materials. Interview key staff using questions like, What are unique characteristics of this program? How do these impact youth? When youth finish your program what do you want them to achieve or have? How would you measure that? These perspectives can assist you with understand factors that may potentially lead to outcomes. Review program materials such as curriculum, lesson plans, and activities, and organize them into common themes. For example, I read through the lesson plans of one program and identified managing conflict as a consistent theme and used this to help develop program outcomes.
I also used the findings to identify literature to support the program’s efforts. For example, I framed the activities of the program using research in positive youth development.
Hot Tip: Use visual aids. Use a visual aid to draw connections between program characteristics and outcomes. This can be created using basic shape features in Microsoft Word or SmartArt. I found that this visual aid helped the program think about how to achieve its objectives and communicate its model to external constituents.
Hot Tip: The “if” and “then” logic. It appears that everyone talks about using logic models in evaluation. Using the information you collected from program staff, materials, and visual aid, develop a series of “if” and “then” statements. For example, IF program X provides activities in conflict management and resolution THEN youth participants will improve their ability to manage and resolve conflict.
Hot Tip: The evaluation plan. Here is when you start gearing the program and yourself up for the evaluation. This can be similar to logic model development; I integrate each of the previous steps in this phase and include objectives, measures, participants, analysis, and outcomes. It not only helped me in thinking through the evaluation, but assisted the program in communicating their efforts to external constituents.
Rad Resource: Program Theory and Logic Models by Amhert H. Wilder Foundation
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to email@example.com . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.