Hello, AEA365 community! Liz DiLuzio here, Lead Curator of the blog. This week is Individuals Week, which means we take a break from our themed weeks and spotlight the Hot Tips, Cool Tricks, Rad Resources and Lessons Learned from any evaluator interested in sharing. Would you like to contribute to future individuals weeks? Email me at AEA365@eval.org with an idea or a draft and we will make it happen.
Hi. I’m Ron Schack, Ph.D., the Managing Director of The Charter Oak Group, LLC in Connecticut. I also teach performance management accountability and advanced quantitative methods in the University of Connecticut School of Public Policy.
Infographics—“Danger Will Robinson!”
A note about infographics. Infographics have become very popular, for good reason. They can send unambiguous signals regarding very specific data findings. They give “just the highlights” of much more elaborate and complex analyses in easily digestible form. This is no doubt useful, and can be important in fostering data democracy. But—you knew there was a but, didn’t you? The same thing that makes infographics easy to understand and powerful, is something that can be abused. Infographics are by their very nature SELECTIVE about what data are shown and not shown. This very practice is one area where statisticians and other data-promulgators have earned a bad name. Infographics tend to ignore or minimize data that might not be consistent with the primary findings being shared, and it is usually not desirable to create infographics with lots of caveats. Infographics tend to suggest the world is less complex than it is…and therefore should be viewed with caution. I do not contend that infographics or the infographic approach are never appropriate or useful, but I do think great care must be taken not to fall into the data selectivity without transparency trap.
Data Storytelling—Sometimes There Is Not A Story To Tell—Yet.
There are a lot of good resources out there that provide excellent guidance on developing and telling data stories. Just a couple thoughts on this one. First, I want to applaud the recent emphasis on telling good data stories. This is one remedy to the “let’s spit out and display a thousand tables and charts and call it analysis” problem. However, I do want to note that sometimes there is not a story to tell, yet. When you are first collecting data on a program, or a social problem, or almost anything, it may be a while before you have enough good data to tell a story. It may be appropriate, in those instances, to make the lack of data the story, and advocate for further data development. Other times, the story is ambiguous, and few of us are good at telling ambiguous stories where we have to say, “it could suggest this, but it could also be that, but maybe it is that.” Doesn’t sound like much of a story, does it? The caution here is to not let our need to tell a succinct and compelling data story lead us to OVER INTERPRET the data we have, or lead us to again fall into the data selectivity without transparency trap.
This is not to say we shouldn’t keep trying to develop good data stories. When done well, without over interpretation or selectivity without transparency, a good data story, supported by appropriately calibrated data displays, can be a very useful and powerful tool. And I am the first to admit that some selectivity–directing the reader/audience to specific data points or trends–is necessary and appropriate. Just be intentional—and transparent—in your choices when you create your storyboard and get ready to tell your data story.
Rad Resources
I highly recommend Storytelling with Data by Cole Nussbaumer Knaflic for learning more about telling data stories.
Check out Mark Friedman’s description of the importance of creating a data development agenda in his book, Trying Hard Isn’t Good Enough (2005).
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.
Hello Dr. Shack! I appreciate your points here in this article. Especially the suggestion to “make the lack of data the story”.
I’m learning that in research we have some freedom in that role to choose topics paralleled with our interests- and speaking for myself here, therein may lie a “danger” in a desire (why I chose the topic) vs what is being indicated in the data. For instance, if another researcher looked over the same qualitative data- would their findings match mine? Or how closely? We learn well that way, in teams, but online learning has it’s limitations. Your article has me making a mental note to utilize the discussion forums more deliberately.
In preparation for future careers most of us (students) are utilizing research skills for the first time in formality. So we may have “much drive”, referentially I mean, yet a need for guidance in stepping back from collected data to weigh how our own narrowed perspectives can distort.
I was unfamiliar with the title reference here (“Danger Will Robinson!”) until I just looked it up- great title by the way! I know that I felt that internal struggle or perhaps discomfort in relation to these two topics on occasions when I myself was trying to make sense of information collected- although you’ve articulated it here so well. So I thank you! I needed that word- ambiguous.
Hi Ron, I enjoyed the reading. It is the first time I come across the “data selectivity without transparency trap” term. Do you have any good examples of authors being transparent about why/how they chose the data they present in infographics?