Hello fellow evaluators, I’m Kelly Hannum President of Aligned Impact and a consultant with the Luminaire Group. As someone who studied research methodology, it’s fascinating to watch as we individually and collectively struggle with what’s true and what matters in an age of “fake news” and “post-truth.” I’m disheartened to see mostly shallow attempts at answering these questions and the slapdash manner in which we engage in these discussions.
Figuring out what’s true and what matters across different perspectives and different values is particularly difficult and is often uncomfortable. It can also take a lot of time and no one is going make you do it. However, if you don’t have some sort of strategy for gathering and making sense of information as a Foundation or a Nonprofit, you’re probably wasting resources and possibly creating harm. As contexts shift and new perspectives and ideas come to the fore, we have to continually reflect on what information we’re getting, how we’re making sense of it, what we’re doing about it – all while paying attention to what perspectives and types of information we are accessing and privileging. There’s little hope of doing that well at an organizational level, if we haven’t done our individual work in this area.
I’ve found myself reflecting on the ways in which I get, test, and use information. Research and journalism (sources I rely on for information) are supposed to be bias-free, fair and balanced, but they aren’t and how we engage with them isn’t either.
Lesson Learned: There is bias and error in all information. Understanding how information can be biased is helpful. Equally helpful is understanding the roots of bias within ourselves. We often think of other people deceiving us, but the best place to begin to whittle away nonsense is within ourselves. The more we know about how to gather, interpret, and use information, the less likely we are to get caught up in assumptions, bias, and outright deception.
Rad Resources:
Read and reflect on the common sources of cognitive bias. You can start by using this article to figure out where you may be leading yourself astray: http://mentalfloss.com/article/68705/20-cognitive-biases-affect-your-decisions
Read and reflect on sources of methodological bias and measurement errors. This blog post by Helen Kara offers some great reading suggestions about indigenous methods as well as better understanding the connection between colonization and research methods, while it’s not the only source of bias and error it is one deeply ingrained in many of our approaches: http://blogs.lse.ac.uk/lsereviewofbooks/2017/07/26/reading-list-8-books-on-indigenous-research-methods-recommended-by-helen-kara/
The American Evaluation Association is celebrating Nonprofits and Foundations Topical Interest Group (NPFTIG) Week. The contributions all this week to aea365 come from our NPFTIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.