Hi, my name is Keisha Farmer-Smith from Become, better known as Dr. Kei-Kei! I received my nickname from young people in an afterschool program while conducting evaluation and outcomes work. It is an honor to learn from children and help create quality youth development programs. Artwork is one way to creatively explore program strengths, opportunities, and challenges.
Recently, I had the opportunity to use artwork in evaluating an innovative program, Chicago Pre-College Science and Engineering (ChiS&E). In ChiS&E, “little engineers” aged 6-9 explore their community and have fun! The program’s mission is to increase the number of historically underrepresented African American and Latino students who are motivated and academically prepared to pursue careers in science, technology, engineering and mathematics (STEM). As a part of a longitudinal program evaluation, ChiS&E youth participated in interviews with an art component. Little engineers were invited to draw pictures of their ChiS&E experiences:
1st grade drawings- students demonstrate a strong understanding of experiment steps and the chemical engineers’ tasks when combining components to get a reaction as well as enthusiasm about lab work. The figures in the pictures are smiling and working.
Elijah: We put the vinegar and baking soda in the water bottle, we went outside and put the cork on it the cork exploded off the water bottle.
Evaluator: Awesome! And that’s what chemical engineers do? They do things like that?
Elijah: (nods head)
Evaluator: And what about you, Paola, what did you draw?
Paola: I was trying to put the hand up here to hold it and the bottle was bubbling, and the bubbles that had air on it was putting the air inside the balloon.
Lessons Learned: When evaluating youth programming, art-based activities are an exceptional way to learn because:
- Drawing artwork together builds rapport and engages program participants. Young people may feel uncomfortable talking to unknown evaluators/adults. Art-based activities are a fun, nonthreatening way to break the ice.
- Art is a great communication tool. When young people explain their art, relevant and meaningful information about programming is shared. During this process, it is incredibly important for adult evaluators to actively listen as youth share the intent and meaning behind their artwork.
- Art can be a strength-based tool. When young people make art it affirms their vision, voice, and opinion. As the artist, the young person has power to explore topics that are important to them.
- Using art as an evaluation tool helps evaluators collect rich, interesting and valuable information. Sometimes, it is easier to express emotions and experiences while drawing. Art-based evaluation should always be accompanied by a listening or discussion session.
- Using art as an evaluation tool is not difficult. From stick figures to complicated images, art is fun for everyone! The artist’s skill level doesn’t matter because the story behind the art can still be shared.
Rad Resources:
Americans for the Arts and Boston Youth Arts Evaluation Project demonstrate two of the many resources for using art in youth program evaluation.
The American Evaluation Association is celebrating Become: Community Engagement and Social Change week. The contributions all this week to aea365 come from authors associated with Become. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hi Dr. Kei-Kei,
I practiced as a professional artist for many years, with a BFA, before becoming a pastry chef. I’ve spent my whole adult life in the creative arts, seeing the power they have at relaying information in formats words cannot. But, in my experience outside of the art world and in the context of evidence-based research, art never had a home. I can’t express enough how refreshing it is to come across someone who is actively advocating for the legitimacy of art as a means of gathering information for evaluation practice. I am currently taking an introductory course to Program Evaluation, as part of my professional master of education at Queens University in ON, Canada, and I recently read “Roles for Contemporary Evaluation Practice: Developing Practical Knowledge” by Donaldson and Lipsey (2011). In discussing their coinage of the term “program theory-driven evaluation science” they say that:
“…’evaluation sciences’ is intended to underscore the use of rigorous scientific methods (i.e., qualitative, quantitative, and mixed methods) to answer key evaluation questions. Renewed emphasis on the reliance of evaluation on systemic scientific methods is especially important for overcoming negative images of evaluation as unreliable, soft, or second-class type of investigation.”
Although I wholeheartedly agree with you that art is a great way of gathering rich and valuable information, I can’t help but wonder if the use of art in evaluative practice would feed into this image of evaluation being an “unreliable, soft, or second-class” investigative practice. How would you respond to someone who would suggest that art-based evaluation feeds into this perception? Is the potential of such a negative perception the reason underpinning your statement, “Art-based evaluation should always be accompanied by a listening or discussion session?” I also noticed that your use of art-based evaluation is in gathering information from young participants. Would art-based evaluation work in the same way for adult participants? Would it be as useful?
Thank you for such a scintillating blog post. You’ve given me lots to think about!
Best,
Dina
Dr. Kei-Kei,
Thank you for your great post! I love hearing about the STEM programs being racially inclusive in the case of ethnic minorities. I think art is a wonderful way to discuss the progression of a program and I imagine that the children may really surprise evaluators at times with what they are able to articulate in their drawings. I would be very interested in reading more research involving children using artwork to communicate the effectiveness of a program or what they have learned.
Thank you for sharing.
Hello Dr. Kei-Kei,
I congratulate you on the work that you are doing and the way in which you meld program evaluation and artwork. The experiences you wrote about with first grade students truly resonated with me as I am a Grade 1 French Immersion teacher and have also seen first-hand the benefits of using artwork as a tool to demonstrate learning, understanding, and knowledge in the classroom. It is fascinating how as adults, we might see one thing and when a child takes to explaining their work, a whole new picture is illuminated. Given that students in my class are learning a new language as well as learning new subject-material in this second-language concurrently, visual representations as a form of expression are so helpful to bridge the connection between the new and unfamiliar. I whole-heartedly agree with your sentiment that, ‘Art is a great communication tool’. I especially like how you conclude that, ‘Using art as an evaluation tool is not difficult’. I wonder how adults would take to this method in a program evaluation setting. I know that this may at first sound a bit primary however, it reminds me of Michael Quinn Patton’s use of children’s stories to open up evaluation dialogues with an adult population. I am wondering if you have ever used artwork with an adult population in an evaluation setting and if so, what were the impressions and results?
Thank you for the work that you do. Wishing you continued success!
Sincerely,
Christina
Hello Dr. Kei-Kei,
Thank you for sharing your article on Using Art in Evaluation. I currently work with elementary aged children in international schools and the drawings / examples you included in the article follow a very similar path to my day to day interactions with children.
I have to agree that art is a great tool evaluators can use with young people to help extract information in a comfortable and meaningful way. I was curious to the types of questions you would ask students to judge the outcome of the program, and how their responses would be interpreted. For instance, using your example from the article, should students be required to know the term ‘chemical engineers’? If so, would the question – ‘Awesome! And who conducts these types of experiments?’ – better suit evaluating the outcome?
I only ask as I am curious about what expectations ChiS&E has on students in various grade levels, and whether you used these expectations to complete the outcome evaluation. I find that even though I have students drawing out their thinking all the time, it is finding the right questions to ask in order to judge whether they have retained the desired outcome we have set for them.
On another note, does ChiS&E evaluate how many participants in the program later pursue a career related to STEM fields? After looking through their website, it seems like such an innovative program that can have a tremendous impact on the lives of participants.
Thanks for sharing!
Best, Jared.
I love the pictures, thank you for sharing Dr Kei-Kei. We have used photographs in evaluations in schools. eg we asked students to take a photograph of what helped them to learn. One student set up 2 different chairs and 2 different stools in a circle – her explanation: “it shows we’re all different, have different ways of thinking and can still work really well together, learn from each other and help each other”. The school vision was about collaboration and respect – so we knew it was alive and well. In schools not so strong on vision and empowering learners – the students have taken a picture of their teacher!
I love that example Barbara! Thank you for sharing it! Using photography is an excellent way to engage youth.