My name is Susan Kistler, the American Evaluation Association’s Executive Director, and I contribute each Saturday’s post to aea365. One hot topic for me this year is data visualization – representing data in ways that are accurate, accessible, and appealing. My very first aea365 post identified resources for those with interests in data visualization – including the classic Periodic Table of Visualization Methods. Today, I want to provide an update.

Hot Tip: Stephanie Evergreen (who gave Great Tips for Graphic Design on July 16) is working to bring together evaluators, and those working in related disciplines, who have an interest in data visualization and reporting in hopes of forming an AEA Topical Interest Group. The group would strive to build capacity and expand the knowledge base in the evaluation field in order to expand stakeholder understanding, improve interpretation, and increase use of evaluation results. As a starting point, Stephanie is building an emailing list of interested individuals and if you would like to be on the list, add a comment to this post (click through back to the website if you received this via email). She’ll also be hosting an informal meeting at Evaluation 2010!

Rad Resources: I attended a presentation in Boston given by the wonderful team at juice analytics as part of their Viva Visualization tour. The presentation was free, definitely worth the 90-minutes of my time, and gave me great ideas for improving my own reports. They’ll be coming to Washington on September 16 if you are in the area. If you aren’t in the DC area, and even if you are, you can learn from their blog – some of the best content can be found on their visitor’s guide and you can subscribe from that page as well. A couple of my favorites? Check out the post on Lightweight data exploration in Excel (under Excel Tricks) to make super-easy inline bars and  Stimulus Bill Explorer (under demos) to see an interactive tree map in action.

Rad Resource: The vizthink group on LinkedIn (you’ll need to join LinkedIn if you aren’t a member but it is free) is a great place to learn from others, post questions, and gather feedback and suggestions.

Hot Tip: Nancy Duarte, author of slide:ology, a book recommended by John Nash in his April 9 aea365 post on Creating Outstanding Presentation Slides. Nancy has a new book coming out – resonate: Present Visual Stories that Transform Audiences due on September 28. She has also created a series of short new videos on creating outstanding presentations, some of which may be found individually on Amazon, but as a set they were uploaded just this week to Vimeo for free viewing.

The above represents my own opinions and not necessarily those of AEA. This contribution is from the aea365 Tip-a-Day Alerts, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

data visualization, Graphic design, graphics, presentations

My name is Monica Hargraves and I work with Cooperative Extension associations across New York State as part of an evaluation capacity building effort in the Cornell Office for Research on Evaluation (CORE).  My work with Extension is shaped, in part, by insights we gained through a Concept Mapping research project we did in late 2008.  We wanted to explore, from practitioners’ perspectives, what factors contribute to supporting evaluation practice in an organization.

We used Concept Mapping software from Concept Systems, Inc. to gather ideas in response to this prompt: “One specific thing an Extension organization can do to support the practice of evaluation is …” Contributors included county-based educators and Executive Directors, as well as state-level Extension administrators and Cornell staff.  The raw ideas were pared down to a working set of 80, and then participants sorted the ideas into clusters and rated them on two criteria: potential for making a difference, and relative difficulty.

The individual ideas become points on a “Cluster Map” that gives a visual representation of how participants conceptualized the patterns and themes in ideas (see below. For information on the Concept Systems technology and the statistical techniques that underlie it, see www.conceptsystems.com.)  The ratings are useful for thinking strategically about what to do give priority to when trying to improve and sustain evaluation practice in organizations.

Rad Resource: For more detail on the study, including a handout with the individual idea statements and their ratings on potential difference, see http://core.human.cornell.edu/AEA_Conference.cfm#2008

Cluster Map of Ideas in Response to the Prompt: “One specific thing an Extension organization can do to support the practice of evaluation is …”

Lessons Learned:

• Technical assistance and training are not enough! The top-rated cluster in terms of potential for making a difference was “Communicate the Value of Evaluation.”  The ideas there included educating organization leaders, staff, and volunteers on the importance of evaluation (not the how-to), using evaluation results well and demonstrating how they lead to better programming, having an evaluation champion in-house, making evaluation results easy to understand and user-friendly.
• Communication is important. Communication should be used to motivate evaluation and build organizational commitment to it, and as a practical tool for sharing what works, fostering collaborations, and saving time.
• Leadership and Structure matter. The second and third most important clusters were “Set Expectations and Requirements” and “Integrate into Organization Structure”.  Respondents wanted clarity and consistency, and to have evaluation woven into a wide range of organization functions and practices.

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

cluster map, data visualization

My name is Nina Potter and I am currently the Director of Assessment for the College of Education at San Diego State University. I’d like to share a little about a tool we are using for data visualization.

One of my responsibilities is to work with program directors and department chairs to evaluate academic programs across the college’s eight departments and 30+ programs. Our programs vary greatly in size and each has its own goals and student learning outcomes. Plus, we have some common goals across the college. We wanted to have a common tool that would allow us to share data across the college, but it had to be very flexible in terms of the kinds of data that it could handle as well as the kinds of reports that it could generate. After a lot of exploring, we chose Tableau.

Rad Resource: Before coming to SDSU, I had never heard of Tableau, in fact I had not heard the term “data visualization tool.” First I will tell you what it is NOT. Tableau is not a tool for data entry. You use Tableau to access data from other data sources such as spreadsheets or databases. This was important because our programs use many different tools to collect data, from electronic portfolio systems to paper and pencil tracking (we do require them to at least put the data in a spreadsheet). And, Tableau does not do advanced statistics; although it does do simple regression and t-tests. For statistical tests, we still use other statistic packages.

So what does Tableau do? Tableau allows you to link into multiple data sources, and quickly and easily create interactive graphs and charts that are updated in real time as your data sources are updated. It has a variety of choices for visualizations such as tables, line graphs, bar charts, pie charts and geographical maps. With just a few clicks you can easily change the type of chart, add colors, add filters and drill down to data that fits certain criteria. The charts are interactive so that anyone viewing the charts can apply filters and view the data they want to focus on.

For example, we have some assessments that are given across multiple programs. We can create a chart that looks at student progress over time and add filters such as program, gender, ethnicity, and age. A person who is evaluating the program as a whole can compare the results from program X to program Y to see if there is equity across multiple demographic groups. Additionally, a person who is working with individual students can download a list of students who have failed more than one assessment in a given program.

Want to hear more about Tableau from Nina? Join her on April 29 for “Data in, Brilliance Out with Tableau” as part of AEA’s Coffee Break Demonstration Series. More information and registration may be found at http://comm.eval.org/EVAL/coffee_break_webinars/Home/Default.aspx. Free for AEA members!

data visualization, software, technology

My name is Juan Paulo Ramírez and I am a research specialist with the University of Nebraska Public Policy Center. I use Geographic Information Systems (GIS) for project evaluations which have included a broad variety of applications both in the social and physical sciences. Recently a lot of interest has been concentrated on geographic visualization, in particular the integration of geobrowsers like Google Earth with commercial GIS software such as ArcGIS and others. This allows distributing GIS data to many who would not necessarily use GIS software but are consumers of geospatial data. The good news is that the use of GIS software and the data associated to it has become easily accessible to the evaluation community at a very reduced cost and sometimes with no costs at all!

Hot Tip: Check out the YouTube videos posted by ESRI, the California based enterprise that created ArcGIS. Search for “ESRI TV.” If you are a neophyte to GIS and in particular to the ArcGIS family, these tutorial videos will save you thousands of dollars in training. Even if you have experience in using ArcGIS, these tutorials will demonstrate new tricks that will enhance your analytical capabilities. http://www.youtube.com/results?search_query=esri+tv&aq=f

Cool Trick: The newest versions of GIS software, including ArcGIS, have incorporated exporting features for Google Earth format files, known as “KML” format. This will allow you to distribute your GIS information (points, lines or polygons) along with databases to your stakeholders, colleagues, and community in general who do not have GIS software. The only requirement is that your recipients must install the free version of Google Earth in order to read KML format files, and the geobrowser will automatically display all the geospatial information and associated databases that you sent to them (i.e., attached in a e-mail). To see the new features of Google Earth, check: http://www.youtube.com/watch?v=GSuJq4UzkIA

Rad Resource: Want to learn more about GIS and spatial visualization? The following book explores the theory behind geographical visualizations, including examples of map animation, and geovisualization tools, and provides insights to the future development of geographic visualization: Dodge, M., McDerby, M., & Turner, M. (Ed.). (2008). Geographic visualization: Concepts, tools and applications. West Sussex, England: John Wiley & Sons, Ltd.

Want to learn more from Juan Paulo? He is offering a Coffee Break Webinar on Using Google Earth for Evaluation: Applications in Environmental Evaluation and Beyond this Thursday. This is free for AEA members and a paid pass is available for nonmembers. Learn more at http://comm.eval.org/EVAL/coffee_break_webinars/Home/Default.aspx

data visualization, Environmental Program Evaluation, EPE, Geographic information systems, GIS, YouTube

My name is Jacquelyn Christensen and I am an Evaluation Associate at a non-profit mental health agency in Los Angeles, as well as an advanced graduate student in Applied Developmental Psychology at Claremont Graduate University. I work closely with my organization to report program outcomes and monitor service quality. When working directly with staff or clients, I must find creative ways to involve them in the evaluation process and promote accuracy during data collection.

Rad Resource: When presenting information to staff, I often try to make seemingly uninteresting information visually stimulating. For a recent presentation of data from staff feedback regarding new software, I compiled all of the qualitative comments regarding their positive and negative experiences with the software and created a “word cloud” using Wordle. This “picture” acted as the cover page and introduction to the presentation. I received overwhelmingly positive feedback from staff who felt it not only piqued their interest, but accurately represented their thoughts and ideas. www.wordle.net

Rad Resource: Surveys are an inevitable part of my data collection, and, despite clear wording and simple organization, I have found people often read quickly, resulting in a discrepancy between their item responses and qualitative feedback. For Likert-type scale responses on our client (and some staff) surveys, I have placed a smiling face above “Strongly Agree” and a frowning face above “Strongly Disagree” as a visual reminder of the direction of the scale. Since doing this, I have noticed a great improvement in the congruence of the quantitative and qualitative responses.

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

data visualization, survey, wordle

My name is Bianca Montrosse and I am a Research and Evaluation Specialist with the SERVE Center at the University of North Carolina at Greensboro. Since receiving my first Edward R. Tufte book as a gift, I’ve been searching for ways to visually articulate data that are both innovative and informative. Below are some rad resources and a cool trick that I have encountered during this quest.

Rad resource: Two lectures on http://www.ted.com/ are particularly fascinating in terms of conveying complex information in an interesting way. Hans Rosling’s lecture entitled, “Let my dataset change your mindset” uses data-bubble software (www.gapminder.org) about the developing world to address common misconceptions. The lecture can be viewed here: http://bit.ly/HRosling. For those of us who commonly work with data collected over multiple time points, I think the possibilities are limitless.

The other lecture is by JoAnn Kuchera-Morin. In the lecture, she demos the Allosphere which she describes as a new way to see, here, and interpret scientific data. While I suspect the ability to create similar presentations with evaluation data are hindered by a number of factors, what I appreciate about her lecture is that she has incorporated multiple mediums to convey complex information. And, that is the lesson I take away from this lecture and try to apply in my own evaluation work. JoAnn Kuchera-Morin’s lecture can be viewed at http://bit.ly/JKMorin.

Cool Trick: Stuck in a rut visually displaying your qualitative data? Try a word cloud. The one I prefer is Wordle because the software gives prominence to words that appear more frequently. So, words that are mentioned most often appear larger in font than those mentioned less often. http://www.wordle.net/

Rad Resource: Looking for other innovative visual displays to get your creative juices flowing? Check out the human trafficking poster created by Taulant Bushi (http://tinyurl.com/22jkm9) to show the number of individuals smuggled into and out of countries worldwide. Whether you zoom out or zoom it, it’s quite an impressive display of a lot of information. Another example is Good Magazine’s comparison of the first 100 days of 13 U.S. Presidents. I think this one does a particular good job of conveying information concerning a number of different data points over time.  http://bit.ly/1st100days

This contribution is from the aea365 Daily Tips blog, by and for evaluations, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

data visualization

My name is SaraJoy Pond, and I am a doctoral candidate at Brigham Young University.  I’m particularly interested in evaluation capacity-building for social change.

Cool Trick: Chances are, if you’ve worked at all with qualitative data sets, you know the feeling of “drowning in data.” The end of an evaluation engagement often leaves me swimming in reams of interview transcripts, gigabytes of video, and hours of audio (somehow always virtually inaudible) wondering “where do I even start!?”  Wordle is a free web-based word cloud generator, has come to my rescue many times. Simply paste in your transcripts or notes and in seconds, you’re looking at an intuitive visualization of “themes” from the text.

Though the tool itself relies on “quantification” (as compared to true qualitative analysis, my professors inform me) I find it invaluable for wrapping my head around that first stage of qualitative data analysis.

TIPS 2 TRY:

• Sort the responses to a question by stakeholder group, then compare the resulting wordle images to see differences in how each group responded.
• Make periodic wordles of your own field notes to see how your impressions, interpretations and judgments varied or evolved over the course of a study.
• Include a wordle image in your evaluation report to help stakeholders see how the themes you discuss “emerged.”

Wordle is available at http://www.wordle.net/

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

data visualization, qualitative, wordle

My name is Laura Blasi.  I work in the field of assessment and institutional research, evaluate grants in the field of education, and teach research methods to graduate students.  In all cases I am trying to communicate in ways that are engaging but are still have meaning (to them and for me.)  I write a lot of reports, and I always want to convey my findings graphically, and I try to find a way to integrate the images into my writing.  My resource is using Sparklines to help convey visual data.

Rad Resource:  Edward Tufte in his book Beautiful Evidence defined sparklines as “intense, simple, wordlike graphics” – so when you see a graph illustrating the rise and fall of the dollar over several years and that graph is embedded in a paragraph only as high as the letters in this sentence.

For example:

Using sparklines I can show a trend and you do not have to break from reading to find a separate chart or graph somewhere else on the page.  About sparklines and more examples from outside of evaluation here: http://bit.ly/TufteSparklines

I wanted to share this here and would love to see other examples actually made in evaluation reports in our field – maybe knowing the importance and/or the impact would be great, too.  Has anyone done this for a funder or agency?

This contribution is from the aea365 Daily Tips blog, by and for evaluations, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

data visualization, sparklines, Visualization

My name is Susan Kistler and I am the Executive Director of the American Evaluation Association. An avid interest of mine is in data visualization – how to make data more accessible through graphical representation. Please note, the following represents my own interests and not an endorsement by AEA.

Hot Tip: Periodic Table of Visualization Methods: Stuck in a rut visualizing your data? Making so many pie charts you might as well be cooking Thanksgiving dinner? Check out the Periodic Table of Visualization Methods for great ideas on ways to visualize data. It offers approximately 100 examples of data visualization options, arranged thematically as a periodic table. Warning: a printed version doesn’t do this justice. Play with it online where you can mouse-over each cell in the periodic table and see an example to get the juices flowing.  http://www.visual-literacy.org/periodic_table/periodic_table.html#. Also, you may want to look at the research behind it all via this academic paper about the Development of the Periodic Table: http://www.visual-literacy.org/periodic_table/periodic_table.pdf. And for the resources? Four websites for data visualization inspiration:

Rad Resource: Flowing Data (Strength in Numbers): Great examples of data visualization from the author, Nathan Yu, a PhD statistics student with a background in design. He also compiles other data visualization resources and encourages sharing and open-source. http://flowingdata.com/

Rad Resource: Information is Beautiful (Ideas, Issues, knowledge, data – visualized!): From the author of The Visual Miscellaneum, David McCandless. McCandless describes himself as a ‘visual & data journalist’ interested in ‘how designed information can help us understand the world, cut through BS and reveal hidden connections, patterns and stories underneath. Or, failing that, it can just look cool!’ http://www.informationisbeautiful.net/ He also has a great scrapbook on Flickr at http://ow.ly/DaIG

Rad Resource: Infosthetics (Where form follows data.): Maintained by Andrew Vande Moere at the University of Sydney, this site ‘collects projects that represent data or information in original or intriguing ways.’ His examples veer wildly from mapping the impact of global warming to a tour of the brain to stitching travel itineraries on postcards – food for thought! http://infosthetics.com/

Rad Resource: Visual Complexity: Focusing on visualizing complex networks, this site archives and annotates projects from around the world that use systems visualization tools. ‘the project’s main goal is to leverage a critical understanding of different visualization methods, across a series of disciplines, as diverse as Biology, Social Networks or the World Wide Web.’ http://www.visualcomplexity.com/vc/ Enjoy!

data visualization