Welcome to aea365! Please take a moment to review our new community guidelines. Learn More.

Antonio J. Castro on Program Evaluation: More than Just Numbers

Hello. I’m Antonio J. Castro, and I am an assistant professor in the Department of Learning, Teaching, and Curriculum at the University of Missouri-Columbia. I teach courses in qualitative research and have been project director and coordinator for a variety of grant-funded initiatives.

Project directors are constantly tasked with trying to represent the quality of their educational projects and programs to funders, whether they are private institutions or larger agencies. Since most projects are centered on goals that are defined by measurable outcomes, evaluation tends to be focused on quantitative data.

Unfortunately quantitative measures fail to communicate all the benefits that your project might offer. Collecting qualitative data, such as from interviews or focus groups, can help communicate the essence of your project and illustrate its outcomes clearly to stakeholders. Here’s a quick list of ways to collect and incorporate these more personal and descriptive kinds of data for your program evaluation.

Hot Tips:

  • Collect application or entrance statements. You might ask participants about their motivations, hopes, dreams, and desires for participating in the project. These can help demonstrate the characteristics and strengths of the project and its applicants.
  • Interview participants.  Project coordinators can track the progress of participants in their program. One way to do this is to select a handful of participants and interview them about their experiences in the project at different points in their involvement.
  • Collect newspaper clippings, announcements, and other related media.  One grant-funded, we included a video of a local news segment that featured our project participants as part of our annual report. This really helped communicate the impact of our project and allowed our participants to come “alive” for the funders.
  • Collect letters of support from stakeholders. Statements from stakeholder attesting to the impact of the project can show funders that the project has a wide reach in the community. For example, one project devoted to recruiting second career bilingual education teachers for urban schools asked family members (spouses, children, etc.) to write letters about how the program had positively impacted the entire family.
  • Collect anecdotal Stories.  We often hear about participants who overcome difficult circumstances or reached a level of accomplishment as part of our project. Incorporating some of these stories into the documentation makes more concrete the value-add of the project.
  • Administer exit surveys for participants.  In an exit survey, Likert-type items can trace the satisfaction of participants with the project. Open-ended items, such as “What was the greatest benefit you received from participating in this project?” can really highlight the strengths of the project.

The main purpose behind collecting and reporting these more qualitative measures is to convey the quality of the project in a concrete and humanizing way to grant funders.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.