Hello again! My name is Jan Noga here with more tips and lessons learned to help you in evaluating programs serving young children. Today’s post will focus on implementing and facilitating evaluative activities.
Hot Tip: Using pictures, colored discs, shapes, stickers, or other concrete ways to express a response can be a very useful way around the variations present in expressive language. If you use stickers, keep a close watch on them and set some rules for using the stickers (“Stickers only go on the paper”).
Lesson Learned: I have found that smiley/frowny/neutral faces are less reliable than using an indicator that is not emotionally loaded. The context a child brings to the classroom matters – some children may associate frowns with bad circumstances at home (such as anger, dysfunction, or possibly abuse). As a result, they may shy away from frownies or refuse to use them.
Lesson Learned: We learned something very interesting when we were using primary color dots. One of the field data collectors decided to go out and pick up some neon colored dots instead. It totally threw the kids (and data) off. Some complained that the dots were too bright. Others loved the brightness, but spent more of their time playing with the dots than responding to questions. We had to completely redo all the classrooms (using a different data collector), but it provided insight into the power of color – as well as the importance of piloting everything and training and monitoring your team.
Cool Trick: Use a simple grid with big, bold, capital letters or numbers in each box. Give children stickers or dots to respond to questions such as, “I like to come to school.” Children respond by putting a sticker in the appropriate box with different colored stickers representing the answer choices. Of course, you may need to prepare students and always have to keep an eye out for patterning (a child who uses the same color no matter what, etc.). In general, setting the survey up as play helps with engagement and authenticity.
Cool Trick: Make it a game. I print two-sided cards cut from card stock. One side has the answer grid with block capitals in each box. The other side is used to collect demographics. I use the sample shown here to collect each child’s gender, birthday, and initials (the “title” of the book in the dog’s mouth). Children cover the appropriate item with a sticker, and I help as needed. At the end, younger children get a coloring page that has the puppy on it to keep and color in later; children in grades 1-4 get a small toy.
Please feel free to contact me if you’d like more information on survey items, scripts, and the protocol.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to firstname.lastname@example.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.