AEA365 | A Tip-a-Day by and for Evaluators

Oct/15

22

2-for-1 Week: Lisa Hinz with Evaluation and Engagement in Community Leadership Education Part 1 – Measuring Action

Hi, I am Lisa Hinz, Leadership and Civic Engagement educator with University of Minnesota Extension. In my position, I am challenged to both foster and document action resulting from our community leadership education programs.

My colleague Jody Horntvedt and I developed a system to encourage program graduates to identify and follow through on their own action items. Evaluation and engagement are of equal importance for us – like two sides of a coin. I am going to highlight the approach and results from our “Action Items” evaluation. Later, Jody will focus on the engagement side.

Our focus here is the Red River Valley Emerging Leadership Program, a five-month cohort program to develop leadership potential and networks among younger leaders in a large region of northwest Minnesota.

At the final session, participants complete an action items worksheet sharing how they intend to practice their leadership in the year ahead.

Cool Trick: For evaluation purposes, we collect the worksheets, scan them, and mail the original back to participants within one month. This way, participants can keep track of their action items, and we have the data for later use. We enter the action items data into Excel, with each action item linked to the participant.

Cool Trick: We designed an online survey using Qualtrics to find out the extent to which participants followed through on their intended actions. Qualtrics allows personalized surveys in which participants can rate their follow through on each of their own action steps by use of embedded data. Here’s how this looks in Qualtrics:

Hinz1

Lessons Learned: These graduates take action – 84% of the 2012 cohort followed up on at least one action step – and they told us how. As seen below, we learn about the degree of follow-through and get more detail on actions taken (or not taken). We also learned that 73% of these graduates applied things learned from the program in ways other than their identified action steps. Here are some examples:

Hinz2

We found even more impressive results in a repeat of the study with 2013 graduates in which 84% took action and 94% applied learning in ways beyond what they identified.

This evaluation process gives an easy, inexpensive, and systematic way to gather data on graduates’ post-program actions – and stay in touch. The data provides outcome data specific to participants and suggests public value – the indirect impact – on the communities where these emerging leaders are exercising their leadership. It also provides opportunities to foster more value in evaluation through interviews and Ripple Effect Mapping.

Rad Resource: To learn more about evaluation of community leadership education, see the recent monograph Community Effects of Leadership Development Education: Citizen Empowerment for Civic Engagement.

We’re celebrating 2-for-1 Week here at aea365. With tremendous interest in the blog lately, we’ve had many authors eager to share their evaluation wisdom, so for one special week, readers will be treated to two blog posts per day! Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

3 comments

  • Laura · October 23, 2015 at 11:13 am

    Hi Lisa, Thanks for the AEA365! Just a thought regarding your first “cool trick”. If you have access to an iPad or other tablet, there are apps that can scan documents for you, I have used “Scannable”. You can then scan the worksheets right then and there and the participants can walk out the door with them in hand. Saving you a couple of steps and postage. All the best!

    Reply

    • Jody Horntvedt · October 26, 2015 at 2:57 pm

      Thanks, Laura, for your comment/suggestion. One of the reasons why we collect them and return them about a month later is that we want that “reconnection with participants” which is part of our engagement plan! We found that most participants return home, set their binders aside, and don’t start working on their plans right away (they need some time to think about it before applying). So, the follow-up is to refocus them. Maybe our plan needs to include an immediate scan AND emailing it back to them…something for Lisa and I to think about!

      Reply

    • Lisa · October 26, 2015 at 3:10 pm

      Didn’t know this, Laura – thanks for the idea!

      Reply

Leave a Reply

<<

>>

Archives

To top