Hello! My name is Taneisha Lee, CEO of Transformative Research and Evaluation, an independent research and evaluation consulting firm in Atlanta, Georgia. We are currently evaluating a Minority Youth Violence Prevention II (MYVP2) grant-funded program for one of our clients, Clayton County System of Care. Two of the grant’s goals are to reduce youth crime and improve youth attitudes and beliefs about violence. With the evaluation, we seek to better understand youth attitudes about violence, which in turn would inform the organization’s programmatic activities.
In April, our team facilitated 8 focus groups with approximately 100 middle and high school youth to better understand what factors and conditions lead to fights and violence at school and in the community. Inspired by the upcoming Women’s World Cup, we provided each participant with a red and green card. During the focus group, the facilitators read different scenarios: “Someone of the same gender says something disrespectful to you,” “You feel angry about something,” “Someone says something about your mother or a close family member.” Students were instructed to raise a green card if it was okay to fight in the scenario and a red card if it was not okay to fight in the scenario.
My first focus group was with a group of high school girls. After reading the first scenario, a few of the students raised both the red and green cards.
I was NOT prepared for this! I asked the students what raising both cards meant. They responded, “it depends.” After further discussion with the students, I learned that fighting was not black and white. Discussions about each scenario revealed the array of factors that influenced students’ decisions about fighting.
After the first round of focus groups, I conferred with the other facilitators and learned some had similar experiences. We discussed the limitations of established survey instruments and the utility of focus groups for truly understanding students’ experiences and attitudes.
Lessons Learned:
- Stay open and flexible! Instead of forcing students to choose one card, we probed to better understand their experiences and gained a wealth of unexpected knowledge about youth attitudes.
- Engage youth before the focus group. Our semi-structured focus group protocol provided opportunities for probing and for youth to guide the discussion based on their experience. The scenarios for the focus group were developed through a collaboration and listening session with a different group of youth about their experiences with fighting. Youth suggested scenarios we would have never considered.
Hot Tips:
- Think outside the box. Mix things up by incorporating different types of questions and mini-activities during the focus group. We often include a piece of paper and writing utensils and ask participants to writing ratings or words before sharing them with the larger group.
- Thank your participants. We provided each student with a zip-lock bag with the red and green card, pen and paper, and candy. Our students LOVE sour candy!
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hello,
Thank you for sharing your experience as an evaluator for the MYVP2 program. The program context you provided helped me understand the rationale behind the implementation of some data collection strategies such as the green and red cards. I appreciate the ease of using the cards to express emotions and was somewhat happy to read the result of such a technique, as it opened up the opportunity for further clarification of emotions, good and bad, based on the context at hand.
I am currently working on developing a PED for a Scouts Canada program called Kub Kars. The Kub Kars program focuses on children ages 9-12 years old, so your article is relevant due to the common age of the children our programs focus on. Although the Kub Kars program is not centred around youth at risk, the goal of my evaluation is also to understand youth attitudes as related to the Kub Kars program and the impact the program had on them. Of course, this is with the intend to improve the program and improve the short- and long-term outcomes.
The specific area which immediately attracted me to your article is the data collection methodology. I decided to use focus groups with open-ended unstructured questions as a way to gather information from children who completed the Kub Kars program. I noted that you completed a number of focus groups, so I am interested in how many children were included in each group and how and why was the number selected? Also, your recommendation is to engage the youth before the focus group. My plan is to have the same people who were involved in delivering the program run the focus groups. In retrospect, I am wondering if this strategy will impact potential answer bias. Lastly, I really liked your idea of incorporating different types of questions during the focus group. Maybe one way to eliminate potential answer bias is to provide the participants a survey where they can give feedback anonymously. The mixed methods data collection during the focus groups may support and validate the focus group findings.
Thank you for providing your experience. It helped me think of how to better plan the evaluation to ensure meaningful, valid and reliable results!
Jeff
Hi Taneisha,
Thank you for your insightful article. Similar to your previous commenter, I’m also a student at Queen’s University, doing a masters class about program evaluation design. I’m very glad to find your post because it gives me some new ideas about how to include focus groups when evaluating teen programs. One of the ongoing tasks for my class has been to create a program evaluation. I chose an alternate education program out on the west coast of BC for at risk youth. When looking at ways to collect and evaluate data I hadn’t thought of the focus group, but this seems like a very useful method that could deliver excellent data. You mentioned engaging youth before the focus group, this also seems like a very good idea. Building relationships with the participants, or at the least some rapport, could really get the group to open up and share their opinions and attitudes. As an evaluator, is this something that is possible or is time often limited to coming in and doing the work and then leaving? Would an evaluator ever have a program facilitator lead a focus group or work with the evaluator at the same time?
I thought the green card, red card system was a great way to easily identify how people felt, also tying it to a popular sporting event is a smart way to make a connection to the youth. Providing a treat for the group is also nice touch, recognizing your appreciation for their effort. Your advice to stay open and flexible to responses was likely a key to getting data you hadn’t expected. As I prepare to complete my final program evaluation design, I will take these ideas forward and use them. I think a focus group would be an excellent data collection method in the program I am evaluating.
Thank you very much!
Nigel Joseph
Hello,
My name is Kat and I am a master’s student at Queens University in Ontario, Canada. I am taking a course in Program Inquiry and Evaluation. I am very interested in topics related to evaluating youth programs, as they are a unique target group. I feel this is a special group because youth can be such a tumultuous stage of life for many. They are searching for self-identity and a sense of belonging. Youth is a time of exploration, sometimes in harmful and dangerous areas. It is a time to discover preferences, socially, emotionally, physically, academically, and more. This stage of life is often a turning point to make productive choices that lead to successful careers as adults, or life altering choices that could lead to closed opportunities in adulthood, especially if it involves crime, violence, or drugs. Hence, thank you for this enlightening article and the succinct bullet point summary.
Your focus groups revealed significant information on the “grey” areas for your topics. Youth do not necessarily interpret events as black and white, as you mentioned. Much of the decision-making is based on the context or circumstances of the actions including the participants involved. Engaging youth in a variety of ways, yes candy and food are great motivators, is important to consider. Youth are more eager to share in discussions, rather than in writing tasks with “fixed” or categorical responses. As illustrated, a richer and greater depth of information was shared and collected through focused and guided discussion. Furthermore, being flexible to allow the discussion to move to slightly broader or different topics can also reveal pertinent information, because it reveals underlying motives and thinking. I have witnessed this in my experience leading youth.
Thank you for this informative article.