Hi! I’m Silvana Bialosiewicz, an advanced doctoral student at Claremont Graduate University (CGU) and Senior Research Associate at the Claremont Evaluation Center. My goal as an applied researcher is to help develop and disseminate “best-practices” for high-quality evaluation of programs that serve children. Today I’d like to share some strategies for collecting valid and reliable data from young children.
Research on youth-program evaluation and child development reveal that:
- Children less than nine years old possess limited abilities to accurately self-report, especially by way of written surveys
- Previously validated measures are not always appropriate for diverse samples of children
Therefore, a critical step in the process of designing evaluations of youth programs is the development and/or choosing of measures that are sensitive to children’s language skills, reading and writing abilities, and life experiences.
Hot Tip: Consider using alternatives to written surveys, such as interviews, when collecting data from children less than nine years old. If written surveys are used, be cognizant of young children’s inability to understand complex questions and accurately recall past experiences. Surveys for young children should be orally administered, use simple language, and use response options that children can easily understand.
Hot Tip: Do not assume that a measure, which has been demonstrated to be valid in a previous study, is appropriate for your participants, especially when the program serves a diverse population of children. The majority of psychological measures for children have been developed and normed on samples of high SES Caucasian children and cannot be assumed to be valid and reliably for diverse samples of children (i.e. English Language Learners, ethnic and cultural minorities, children with physical or sensory disabilities).
Hot Tip: Pilot test your measures, even previously validated measures, before launching full scale data collection to ensure developmental and contextual appropriateness.
Rad Resources: Researching with Children & Young People by Tisdall, Davis, & Gallagher and Through the Eyes of the Child: Obtaining Self-Reports from Children by La Greca are two great books for anyone looking to expand their knowledge on this topic.
Other AEA365 posts on this topic:
Susan Menkes on Constructing Developmentally Sensitive Questions
Krista Collins and Chad Green on Designing Evaluations with the Whole Child in Mind
The American Evaluation Association is celebrating Ed Eval TIG Week with our colleagues in the PK12 Educational Evaluation Topical Interest Group. The contributions all this week to aea365 come from our Ed Eval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hi Silvana,
I enjoyed reading this post as it directly connects with a course I am taking on program inquiry and evaluation. I am currently evaluating a youth reading program at a local library and I need to determine the best data collection methods connecting to my choice program. I found your post specifically, because my program involves children and I think it is important to understand how to make age-appropriate survey or interview questions that will yield valid and reliable results.
I am happy you pointed out that we should be aware of the diversity of our program participants and not to assume that a previously used measure would be the best choice for every program.
Do you think it is best to include parents/guardians in data collection when children are involved? Whether that is to provide parents/guardians with their own survey based on the program that their child participated in?
Your opinion would be greatly appreciated.
Thank you so much,
Morgan
Hi Judy,
Most of the evaluation work that our team does with children is in after-school programs. Each evaluation is reviewed by our school’s IRB board before we launch.
The parents of our participants sign a consent form when they enter the program and the students who participate in our focus groups sign an additional assent form. We have found it helpful to include these forms in the enrollment package when students join the program.
If you’d like more details on this process, please feel free to get in touch!
I’m curious about your approach to including children directly in your evaluation project vis-a-vis informed consent and the IRB process. It seems like there’s another blog that could be written extending your methodologic ideas to including children which still necessitates the inclusion of adults/parents in terms of consenting subjects when an IRB is needed for the project.