AEA365 | A Tip-a-Day by and for Evaluators

Feb/17

5

Internal Eval Week: A tool for helping staff translate results into action by Jay Szkola

Hello!  I am Jay Szkola, senior program analyst in the Strategy, Evaluation and Learning Division at Good Shepherd Services. As an internal evaluator, I have the opportunity not only to evaluate the programs in my portfolio, but also assist staff in using data in their day-to-day work with participants. This doesn’t have to involve a great deal of complexity, and can be as simple as developing tools that translate data into an accessible format.

Welcome to internal evaluation week, hosted by the Internal Evaluation TIG! Thanks to all of the internal evaluators who contributed to this week’s blogs and to Annie Gleason (Good Shepherd Services in New York City) for coordinating the work! This week is dedicated to sharing insights and tools from evaluators across the country (and Canada!) about how they help organizations to put data and evaluation findings into action. Blogs this week offer a broad range of suggestions for helping stakeholders to use data, drawing on technology, data visualization, communication and framing, and participatory methods.

All of the community-based youth justice programs at our agency use the Positive Youth Development Inventory as a tool to help evaluate our impact on key youth development constructs, such as friendship and future orientation. We use the PYDI to assess our participants at baseline, and gauge pre-to-post changes upon program completion. These insights are useful for program planning and advocacy.

I also wanted to make sure staff could use the results to inform their work with participants directly, but initially experienced limited success with this goal. One barrier was that while staff liked the survey, they were unsure how to translate individual survey responses into insights and action.

Cool Trick:

In response, I constructed a simple Excel sheet to help with this translation.  Program staff enter in participant survey responses, and get out easy-to-read graphs showing how the participant scored in each construct. As staff enter the data, a table aggregates each response by construct, which then links to the graphs.

Program staff and leadership liked the tool so much that when the NYC Department of Probation adopted the PYDI as a measure, the Excel tool was shared and is now used by other youth justice programs in the city.

(click for larger images)

PYDI Template Input

PYDI Template Input

PYDI Template Output

PYDI Template Output

Lessons Learned:

Offer a guide. For those who do not have a high comfort level with data, colorful graphs alone will not bring about understanding. Included on the Excel tool is a brief, jargon-free definition, a sample question, and a list of questions for each construct on the survey. 

Think about multiple uses to get maximum impact. Whatever the data source (surveys, administrative data, etc.), talk to staff about how they could use the data. Once staff in our youth justice programs began seeing the PYDI as something they could use in day-to-day practice, it became less of a task to be completed “for the evaluators” and instead, a shared project.

The American Evaluation Association is celebrating Internal Evaluation (IE) Topical Interest Group Week. The contributions all this week to aea365 come from our IE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

1 comment

  • Nickesha Francis · February 6, 2017 at 11:20 am

    This is awesome and very helpful tips to work with program staff. Thanks Jay!!

    Reply

Leave a Reply

<<

>>

Archives

To top