This week, the members of the Graduate Students and New Evaluators (GSNE) TIG share various tips, tricks, resources, and points of view that can be helpful for students and new evaluators. We hope both evaluators, new and old, will review this material and share the resources and stories with each other.
-Crystal Luce, GSNE Chair
Hey y’all! I’m Briana Ford, MPH, GSNE TIG Co-Chair, and a public health researcher and data analyst working out of my hometown, Columbia, SC. I joined AEA to learn about evaluation methods and enrich my quantitative methods with wisdom from the field. I feel responsible for making data accessible and relevant for program advocacy. My education and experience are in public health, policy, healthcare research, and cognitive science, which is the topic of this blog.
Hot Tips
Long ago, I was a psychology undergrad on the vanguard of some admittedly niche psychology research on cognition. Ten years later, I endeavor to justify earning my B.A. with this blog. I was inspired by Eval 2023’s theme, ‘The Power of Story’ and Sarah Mason’s article in AJE’s fall issue. Hearing why AEA members advocate for ethical story-telling, I was reminded of that research experience, measuring variation in humans’ very thoughts. We might try exploiting that research base and its methods in two ways:
- A conversation facilitation tool to test and interrogate conceptual mis/alignment with participants and partners.
- By using lessons learned from psychology research, we may be able to lower invisible, mental barriers between the stakeholders with power, the message-bearing evaluator, and the communities in need who often are otherized by systems of oppression and the insidious influence of implicit bias.
Why Evaluators Need Stories
Many of us are anxious to overemphasize quantitative metrics that can’t reflect the humanity they represent. Eval 2023 focused on using stories as a reporting tool, understandably so. How can decision-makers really understand those human stories with nothing more than attendance trackers and pre/post assessments?
I think of data as the “bones” of a program. Data will likely remain the core of the stories we must tell, and we’d risk devaluing the profession without it. I admit, though, that bones only say so much about a whole being. It turns out dinosaurs looked different than we previously thought! It turns out, the data was imperfect AND incomplete; who can be surprised?
The ‘Power of Story’ is the power to describe a program’s “flesh and blood.” Stories provide insights into the program’s overall health – its connective tissue and organs – by relating material conditions and context to program outcomes. For distant leaders in positions of power, it’s too easy to be affected by assumptions about social programs and the communities that need them. With only performance metrics – with only the bones – to judge, who knows what really drove a program’s success metrics? We know, and we can help others see the same picture, that is if we frame it right.
Crafting Stories is an Art. But, it’s also a science.
Cognitive science is concerned with understanding the internal (nature) and external (nurture) influences on how people understand, feel about, and remember information. Implications for social programs’ issue communication are frequently and directly used as use cases, which can inform our decisions on message delivery.
Most people in power positions wouldn’t do harm to vulnerable communities purposely. Can we as communicators change their minds without telling them to change their minds? How do you say “Don’t think lowly of this community.” without risking associating it with stigma in the process? We might learn to strategically integrate narrative frames that emphasize and support ideas that would not be understood the same without context, a bone with no flesh. Narrative framing brings the audience closer to you, metaphorically, by:
- Dampening latent stereotype strength, temporarily clearing the mind of biased junk
- Priming audiences for the story you want to tell them, rather than a story they feel they’ve heard before
Post Script: Get Involved!
Another way evaluators might incorporate metaphors is in qualitative research. To check in with program partners, test your assumptions with a metaphor! For instance, you might describe your program experience as “a hike through the woods” (a little challenging, but healthy and fun!) but if your partners would say it’s been “like an Iron Man,” you might want to call your team in and hear more about whatever you’ve been missing!
AEA is hosting GSNE Week with our colleagues in the Graduate Student and New Evaluators AEA Topical Interest Group. The contributions all this week to AEA365 come from our GSNE TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.