AEA365 | A Tip-a-Day by and for Evaluators

TAG | youth participatory evaluation

I’m Amy Campbell, an evaluator at Centerstone Research Institute in Nashville, TN. While I work on several evaluation projects, one of the most rewarding is the Tennessee Healthy Transitions Initiative, where I’m able to work closely with youth and young adults (Y/YA) who have or are at risk of developing mental health or co-occurring disorders.

This year, we had an opportunity to conduct a Youth Participatory Action Research (YPAR) project with a Healthy Transitions Young Adult Leadership Council for the purposes of generating information to inform the services offered to Y/YA in Chattanooga, TN. We were able to engage Y/YA, train them in research methods, and collaboratively develop and implement a research project. Members of our YPAR team presented at Evaluation 2016 and shared how their findings will impact the design of the Tennessee Healthy Transitions Initiative.

Moving forward, this YPAR team will be meeting with Healthy Transitions leaders at the local and state levels to share their findings and collaboratively develop program solutions based on the data. Their findings will also influence the Leadership Council’s actions in the future; they are discussing social media campaigns focusing on issues they identified and other data-informed projects.

Lessons Learned:

  • Stakeholder buy-in is crucial. If you think you might not have this buy-in, you should start by having discussions that try to address this.
  • Y/YA need adequate support and training to be able to effectively engage. Share the expectations you have for your research team early (e.g., commitment, deliverables, etc.), and provide a solid foundation of the basic tenets of research in your first trainings. Use every interaction with your team as an opportunity to teach them about good research design and processes.

Hot Tips:

  • Free food is one of the best recruitment and retention tools you have at your disposal. We fed our team at every meeting, and we offered food for our data collection “event.”
  • Utilize social media and technology! I am located about 150 miles away from our research team, so we had to find creative ways to stay in contact. We used Facebook Messenger to stay connected between meetings, Evernote to keep track of meeting notes and action items, and Google Hangouts to meet remotely when we couldn’t meet in person.

Rad Resources:

The University of California, Berkeley’s YPAR Hub is a great resource for training and preparation exercises for your YPAR teams.

The American Evaluation Association is celebrating Youth Focused Evaluation TIG Week with our colleagues in the YFE AEA Topical Interest Group. The contributions all this week to aea365 come from our YFE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Hi! I am Cathy Lesesne and I work at ICF International doing public health related evaluation and research. My passion is doing work that affects the lives of adolescents, particularly those with the most need and the least voice in how to meet those needs. I do a lot of work in and with schools and school districts focused on optimal sexual health for teens and how to ensure youth have skills and ability to make healthy choices no matter when they decide to engage in sexual activity.

I often see well-intentioned school or school district staff creating solutions for youth and testing them rather than involving youth in solution identification and evaluation of the success. It is clearly easier to retain the power to determine the solutions and to see if they work in the end through evaluation. However, in my own work I have seen the power of youth engagement and involvement in both developing programs and services as well as in helping to evaluate and improve those resources.

Rad Resources: As evaluators, we often have the ability to make recommendations to our clients and partners working with youth AND we have the power to approach our evaluation work with youth in empowering and engaging ways. But we don’t always know how. I highly recommend that you dig into the Youth-Adult Partnerships in Evaluation (Y-AP/E): A Resource Guide for Translating Research into Practice and find your own ways to apply the wide range of ideas, tip sheets, and examples for engaging youth as partners in evaluation. Many of these examples may also help your clients or partners think of ways to better engage youth in the development of programs and services that reflect them and their real interests and needs. If youth are empowered to be partners in developing and testing solutions, they become allies instead of subjects; sources of solutions instead of sources of data.

The American Evaluation Association is celebrating Community Psychology (CP) TIG Week with our colleagues in the CP AEA Topical Interest Group. The contributions all this week to aea365 come from our CPTIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

Hi, I’m Sara Plachta Elliott, Evaluation Fellow at the Skillman Foundation in Detroit, Michigan, through a grant to Brandeis University’s Center for Youth and Communities. As an Evaluation Fellow, I work with Foundation staff to create learning opportunities that inform the work of improving school and neighborhood conditions for Detroit’s kids.In 2010, Brandeis conducted a System of Supports and Opportunities (SOSO) analysis in six neighborhoods where Skillman focuses its grantmaking and changemaking efforts. Data collection involved interviewing youth program and basic service provider staff to assess program offerings, participation rates, program quality, and accessibility.

Below are learning opportunities that were created with SOSO data.

Lesson Learned: Have stakeholders review findings before the final report is prepared.

  • An internal review with Foundation staff helped identify inaccuracies in participation rates and site location addresses.
  • Then two-page SOSO snapshots for each neighborhood were released.
  • Key neighborhood and youth program stakeholders were engaged in a review of findings and lifted up questions about how data were collected. This review process ensured “on the ground” validity of the analysis.
  • For full transparency, the dataset was also released to partners for system planning purposes.

Hot Tip: Create maps or other visual products.

The Foundation engaged its partner Data Driven Detroit to create maps of agency and program site locations. In meetings with stakeholders, we reviewed these maps along with youth population maps. In one neighborhood, we learned that most kids lived on the west side but program sites were clustered on the east side. The collective “ah ha” moments helped the Foundation and its stakeholders work together to fill gaps.

Consider hosting a discussion session and ask community partners and stakeholders to interpret brief data reports and maps. What patterns do they see in the data?

Cool Trick: Engage youth and residents in reviewing data, not just program staff.

In the summer of 2011, a social service agency, Southwest Solutions, organized a community youth mapping project. Youth reviewed the Brandeis SOSO data, then designed and conducted their own neighborhood opportunities survey.

Youth researchers were paid through summer youth employment funding and walked every street in their neighborhood twice, administering surveys to youth, youth program staff, and business owners, as well as mapping vacant properties. They discovered that some local businesses wanted youth to work during school hours, thereby tempting them to drop out of school. The youth also learned that students wanted more college and career preparation opportunities, mirroring findings from the SOSO report that more of these opportunities were needed.

Evaluation reports can sit on a shelf if not accompanied by intentional learning opportunities. As an evaluator, encourage clients to create interactive learning opportunities with short, visual reporting products. Spark their collective learning!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · · · · · · · ·

My name is Rob Shumer and I work at the University of Minnesota.  I’ve taught a few courses on participatory evaluation, some which focus on youth led evaluation.  I am currently involved with a few projects that engage youth in evaluation of their school programs.

Most youth participatory evaluation happens through programs that occur after school or through community-based youth programs.  Tackling the problem of embedding participatory evaluation within the school day, conducted through classes or other in-school efforts, has proven to be a challenge.   Here are a few tips if you are trying to do such work in schools.

Hot Tip: Challenge #1: Time.  One of the major problems for engaging youth in evaluation activities during the school day is finding sufficient time to devote to the effort.  One of the best ways to make time is to include the evaluation work as a part of a regular course.  In one project a teacher is using her English class to teach evaluation as students evaluate some of their school/community programs.  As regular class activities students learn to conduct interviews, write notes, write reports from notes, and learn how to analyze narrative data.  They are also learning how to develop surveys, focusing on the importance of language clarity and precision to ensure the questions they ask are understandable and effective.

In another program we worked outside the class framework, trying to meet with students during their homeroom and extended lunch period.  The project started off fairly well, but as the semester wore on, teachers were reluctant to have students miss class time….and the preparation of students to conduct focus groups (one of the goals of the program) was cut short. So, work with the school staff to identify appropriate meeting times that don’t interfere with the class activities.

Hot Tip: Challenge #2: Academic fit. The pressures of No Child Left Behind evaluation standards creates a challenge for students and teachers.  If the evaluation activities don’t align with skills in English and Math, the work is considered a distraction.  Find ways to integrate evaluation with regularly occurring learning activities to make the participatory work successful.

The good news with school-based youth participatory evaluation is students really get excited about the learning.  In one of the more successful ventures, 9th graders developed a great survey for their program and now want to continue their work into the next year.  They are more involved in the program, becoming partners with their teachers in developing the program for all the students.  Youth participatory evaluation has great potential to motivate students to learn to become critical thinkers….and to become actively involved in their schools.

The American Evaluation Association is celebrating Youth Focused Evaluation (YFE) TIG Week with our colleagues in the YFE AEA Topical Interest Group. The contributions all this week to aea365 come from our YFE  TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

·

Greetings!  I’m Shep Zeldin and I am a professor in the Civil Society and Community Research Graduate Program at the University of Wisconsin-Madison and I’m Jane Powers from the Bronfenbrenner Center for Translational Research at Cornell University.

We are engaged in a long term project to promote and support youth-adult partnerships (Y-AP) in organizational and community change efforts.   The multiple resources that we have developed can be found at http://fyi.uwex.edu/youthadultpartnership/and www.actforyouth.net.  The notion that youth and adults can work collaboratively, as authentic partners, to design and implement programs remains an innovative idea in the United States.  There are, however, many exemplary models of Y-AP and a strong research base has emerged to support its practice.  Y-AP can be a confusing term.  It has overlap with other concepts such as youth participation, youth engagement, and youth empowerment — but is different in key ways.

Hot Tip:  We have learned that the first step toward Y-AP is definition, discussion, and reflection.  Organizations must come to consensus as to why they want to involve young people in program design and evaluation, as well as identify their goals:  where within the organization do they most want Y-AP to exist.  Once this consensus has been built, progress and momentum follow.

Rad Resource:  We have developed two tools to help organizations “make the case” for Y-AP, and to plan and design evaluation efforts:

 Although many organizations are motivated to engage in assessment and evaluation they often do not know where to start; few have staff experience in conducting these activities.  A further challenge is youth participation.  Organizations want to involve youth as partners in assessment and evaluation, but are often unsure on how best to collaborate with youth throughout the process.

Rad Resource:  We have developed a tool kit that guides organizations through all aspects of the organizational assessment and evaluation process from conceptualization, to forming questions, to collecting and analyzing data, to reporting on findings and recommendations.  Youth Adult Leaders for Program Excellence (YALPE) is especially appropriate for programs that actively seek to strengthen program quality and improve their services and include youth as key partners in that process.

The American Evaluation Association is celebrating Youth Focused Evaluation (YFE) TIG Week with our colleagues in the YFE AEA Topical Interest Group. The contributions all this week to aea365 come from our YFE  TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

·

Hi! My name is Krista Collins, and I am an Applied Developmental Psychology doctoral candidate at Claremont Graduate University.  As a developmental psychologist and practicing evaluator, I am interested in how youth-participatory evaluation can engage youth in meaningful activities that will enable them with the tools to promote their own development. Best practices about how involving youth can benefit both our evaluation, as well as support cognitive, social and emotional development for our youth participants have already been shared.  To contribute to this conversation, I want to share with you the developmental rationale for why these activities are successful and how we can implement them into our evaluation work.

Hot Tip:  Provide Opportunities for Decision-Making and Reflection – Giving youth power to make decisions is a great way for youth to improve practical skills that will help them be successful in life.  Evaluators can promote higher-level thinking by guiding youth through decision-making activities that involve goal-setting, time management and problem solving.  Similarly, encouraging youth to reflect about decisions they’ve made supports skill integration, so that the lessons learned will be readily available in the future.  Research has shown that youth who participate in these types of structured activities are more likely to be empowered and demonstrate leadership in other areas of their life.

Hot Tip: Encourage Teamwork and Mentoring – Group work gives youth opportunities to build their social network by interacting with new and diverse people, learn group process skills (e.g. compromise, idea-sharing), as well as discover prosocial values.  Activities that connect youth to positive adult and youth role models can introduce participants to community organizations and resources that share and support their interests.  In addition to expanding youth support systems, opportunities for teamwork and mentoring can promote social competence, self-regulation, and civic engagement as well as build professional skills that will be useful to their future careers.

Hot Tip:  Facilitate Active Learning – Structured activities, such as participatory evaluation, introduce youth to a host of new interests that are often times unavailable.  Giving youth opportunities for active learning allows them to explore their abilities through trial and error, as well as support identity development by realizing their strengths and limitations.  Youth who have engaged in activities that encourage self-knowledge are more likely to self-select future environments and relationships that will promote their success.

Rad Resource: Dworkin, J.B., Larson, R.W., & Hansen, D.M. (2003).  Adolescents’ accounts of growth experiences in youth activities.  Journal of Youth and Adolescence, 32, 12-26.

Rad Resource:  Hansen, D.M., Larson, R.W., & Dworkin, J.B. (2003).  What adolescents learn in organized youth activities: A survey of self-reported developmental experiences.  Journal of Research on Adolescence, 13, 25-55.

The American Evaluation Association is celebrating Youth Focused Evaluation (YFE) TIG Week with our colleagues in the YFE AEA Topical Interest Group. The contributions all this week to aea365 come from our YFE  TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

·

Hi, I’m Dr. Mary Arnold, a professor and youth development specialist at Oregon State University.

We are not lacking great ideas for how to involve youth in program evaluation, but we often struggle with how to actually making it happen! Here are a few starter tips for ensuring success in involving youth in participatory evaluations. Look for more tips from me in an upcoming AEA365 post.

Hot Tips:

  • Make it Fun!!!! And keep in mind that what is fun for adults may not be fun for youth. Teens love opportunities to socialize, be creative and laugh together. They can also be unsure in unfamiliar situations, so don’t underestimate the time it takes to get everyone warmed up and comfortable. I always begin projects with you with some fun and interactive icebreakers that get everyone up and engaging with each other. Even better, ask some youth to help you lead icebreakers.
  • Establish Solid Youth Adult Partnerships (Y-APS). Youth have a lot to bring to the table in participatory evaluation projects, but they can’t do it alone. Without proper adult support, youth-led evaluations will not succeed. However, they will also be unsuccessful if adults are too heavy-handed, and view themselves as “in –charge.” Striking the perfect balance of power and partnership between youth and adults can be difficult to do, but not impossible. The key is to provide training for the youth and adult partners. There are several curricula available to help with this. Reflect and Improve: A Toolkit for Engaging Youth and Adults as Partners in Program Evaluation, developed by the Innovation Center for Community and Youth Development is a particularly good resource for creating strong Y-APS for evaluation.
  • Engage Youth Deeply Participatory evaluation is about empowerment. When we engage youth in evaluation we are inviting them to question, analyze, and respond to issues that matter to them. Inherent in this process is the invitation for youth to think critically about issues that impact their lives. When I work with youth I teach them to think about the causal roots of problems, as well as the contextual influences that make the problems unique to their lives. When I first started doing this I worried that I would lose their interest because I was asking so much of them intellectually. When I saw the glazed looks I worried it felt too much like school. But I waited. And before long, youth started responded to my probing in thoughtful and provocative ways. Then pencils came out, and notes were taken, heads nodded, and new awareness took form. I have learned not to be afraid to present some challenging content and allow time for it to simmer.

The American Evaluation Association is celebrating Youth Focused Evaluation (YFE) TIG Week with our colleagues in the YFE AEA Topical Interest Group. The contributions all this week to aea365 come from our YFE  TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hi, we are Jane Powers and Christine Heib from Cornell University.  We have been collaborating with a community agency to assess the needs and numbers of homeless youth in the county.  In order to study this invisible sector of the youth population, we engaged a group of homeless youth as research partners.  They helped develop the survey, recruit and interview subjects, interpret findings, and made presentations to key community stakeholders.  Here are some of their lessons learned:

Hot Tips: Connecting with Hard-to-Reach Youth:

  • Go to where youth hang out: “Since we have been there ourselves, we know where to find homeless youth. We are connected to these networks. Knowing one person can open you up to a whole group–people who see what is going on and want to participate.”
  • Offer incentives:  “Offering a food coupon for the interview worked well: people are hungry and did the survey for a free meal.”
  • Stress the “why” of the project: “Homeless youth don’t like how they are living. We tell them how this information will be used and some of the concrete changes that have taken place because of this project, like more housing options.”
  • Emphasize confidentiality: “Homeless youth are worried about being judged. It helped to remind people ‘No one will know who you are – they just need to know what help you need.’”
  • Be open:  “It helps to say that you have been homeless yourself. This makes them more comfortable and willing to be honest; they know they won’t be judged for their responses. Be kind, be respectful, listen well, don’t be judgmental, show that you care.”

Hot Tips: Engaging Youth as Research Partners:

  • Make it fun: “Youth are very responsive when they are excited to be a part of a project and have a good time while doing it.”
  • Involve youth from the start:  “Since we helped design the survey, we felt that it was our project:  these were our questions, not just ones we were told to ask.  We really cared about the answers.”
  • Help them see the benefits: “Going through the IRB training was a self esteem booster; it gave me confidence that I could do this.”  “I learned how to make people more aware of their problems and who to talk to for help.”  “Now I can see that if people work together they can make changes; I used to think no one cares – not now!”

Rad Resources:

Project Final Reports:

For resources on Youth Participatory Evaluation, you may find the following useful:

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Archives

To top