AEA365 | A Tip-a-Day by and for Evaluators

CAT | Data Visualization and Reporting

My name is Elissa Schloesser with Visual Voice, I am a freelance graphic designer specializing in infographics, data visualization and reporting. I enjoy making complex information more understandable and engaging. I have partnered with several evaluators to help visually communicate evaluation methods and findings.

Below are a few techniques I use to make logic models, theory of change and other process diagrams more visually appealing and digestible.

Hot Tip: Start by considering your diagram’s purpose and audience. Edit content accordingly.

Is your diagram intended to be analyzed up close by your reader, or is it intendd to provide a visual overview of your model or process? If its purpose is to be used as a summary, only include the most important and relevant information.

Hot Tip: Establish a hierarchy of information and apply a consistent design style to each level.

Not all information should get equal visual weight and real estate. The main concepts and connections should be the biggest and boldest, while the supporting details should be formatted to be smaller and lighter. Establish a design style for each level of information and make sure it is applied consistently throughout. This is especially important when you are working with a diagram that has lots of layers of information.

Hot Tip: Use color to enhance your diagram, not make the diagram.

I like to think of color as a bonus feature in any diagram. Selective color use can help emphasize connections in your diagram (but try not to get carried away). If you use every color in the rainbow, it tends to be less effective. Additionally, I like to test to see if my diagram is still understandable in grayscale even if it will most likely be viewed in color. If the diagram is not understandable without the color, go back and readjust the line weight, headings or iconography.

Rad Resource: Below is a sample process diagram to help illustrate the points above. It is similar in structure to what you might use for a logic model. This diagram could be recreated in Word or PowerPoint in a table with invisible borders.

Rad Resource: For more advice on how to make your diagrams more digestible, check out this blog post by Grank Denneman – “10 guidelines for creating good looking diagrams”.

The American Evaluation Association is celebrating Data Visualization and Reporting (DVR) Week with our colleagues in the DVR Topical Interest Group. The contributions all this week to aea365 come from DVR TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Hello good people! My name is Robert Perez and I am a research assistant at Hamai Consulting and data analyst at Youth Policy Institute. I am responsible for cleaning and analyzing data from various sources and designing engaging ways to communicate findings using reports, dashboards, and infographics.

I spend some time at the beginning of the year planning out projects for my department using a number of different tools. One tool I love to use is the gantt chart. I searched for a way to automate the conditional formatting process of coloring each of the representative time units and stumbled upon a template from Chandoo.org.

(click for larger image)

This particular gantt chart allows the user to enter the starting week, project duration, and completion status, which will automatically populate the lines with colors depending on the department or project name. Formatting this chart took some “Excelbow” grease as the original chart did not have the option to colorize the tasks based on the department or project.

Updating tools like this gantt chart regularly will encourage its use and keep it from being lost in virtual oblivion. Keep in mind the technical skill of your audience and communicate with them during the design process to ensure that what you are creating will be of use to them.

Hot Tip: If you are having trouble understanding the function of a formula, I find it helpful to break out the formula into its component parts and paste each component into their own cells. This way, I can determine the return value of each nested formula, which gives me more context around how each component works together.

Hot Tip: No matter how often you design a visualization or a tool, the question of “who is your audience” will always come up.  Harvard Business Review offers some definitions of audience categories that might help with your design process:

Novice: first exposure to the subject; doesn’t want oversimplification

Generalist: aware of the topic, but looking for an overview understanding and major themes

Managerial: in-depth, actionable understanding of intricacies and interrelationships with access to detail

Expert: more exploration and discovery and less storytelling with great detail

Executive: only has time to glean the significance and conclusions of weighted probabilities

Rad Resource: Sometimes I need a bit of inspiration when designing my visuals. Fortunately, there is a veritable bevy of resources online from which we can fuel our creative engines. One of my favorite sites to visit is Contextures.com. The site caters to Excel novices, offering lessons about conditional formatting to VLOOKUPs, and those more experienced learners with lessons about automation using VBA. Of course, there is also Chandoo.org, a site that has one of the friendliest community forums ever.

The American Evaluation Association is celebrating Data Visualization and Reporting (DVR) Week with our colleagues in the DVR Topical Interest Group. The contributions all this week to aea365 come from DVR TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

 

No tags

Hello! I’m Julie Lamping, a research analyst at Harper College in Illinois. A lot of what I do is extracting data that is needed, formatting it into tables or charts, and providing a basic analysis to be used during decision making at the College. No matter the project, everything gets thrown into Excel at some point.

I absolutely love pivot tables and charts! I use them for simple validation, creating crosstabs and charts, and building dashboards. Excel has its limitations (*cough*itdieswhenthereistoomuchdata*cough*), so pivots aren’t miracles for everyone (just maybe for some of you).

Start with [fairly] clean data in Excel. Go to the INSERT tab and select PivotTable. Boom, now you have a pivot table (seriously, that’s it). Like everything Excel spits out at first, it is ugly. PivotTable Tools tab will help you design the table to look however you want. You can even group items together. LifeProTip: have your color palette ready by creating a custom style theme in Microsoft.

We’re data visualization people, so if you go back to your sheet with your data, Insert a PivotChart the same way you would a table (hint, it’s usually located by the other charts under the INSERT tab).
Format that bad boy with all the data visualization standards and skills you have. It looks so good (probably).

But say youwant to give someone else the option to filter the chart how they want – we can do that! While the PivotChart is selected, head on over to the ANALYZE tab and hit “Insert Slicer”. Select the fields from your PivotTable you want and click OK. LifeProTip: I use this function when creating dashboards in Excel.

Any selection on the slicer will filter your PivotTable and PivotChart. Hint, these slicers can also be customized to be appealing – I usually make mine gray but Harper’s blue when selected.

Extra fun dashboarding – you can move your PivotTables and Charts using the Move Chart function. If I’m creating a dashboard in Excel, I move my final Charts and Slicers over to new sheet (usually renamed DASH for my sanity) and hide the rest of my mess.

TL;DR: Select the first cell in your excel table and insert a PivotTable or PivotChart. Format and modify as needed. Insert Slicers if you are always extra like I am, then move to a separate sheet for quick dashboard.

Happy Excel’ing!

Rad Resources:

For newbies to Pivots in Excel: Ann Emery’s Introduction to Pivot Tables

Jon covers finalizing the dashboard: Youtube Video on Excel Dashboarding by Jon from Excel Campus

All things dashboard’ing in Excel: SmartSheet’s Post on Excel Dashboards

Shameless plug: My [sassy] tutorial

The American Evaluation Association is celebrating Data Visualization and Reporting (DVR) Week with our colleagues in the DVR Topical Interest Group. The contributions all this week to aea365 come from DVR TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

No tags

Greetings from Minneapolis, Minnesota! I am Angie Ficek, a Program Evaluator at Professional Data Analysts (PDA), an independent consulting firm specializing in evaluating public health programs. We wanted to share three data viz related features in Office 2016 that are easy and helpful for spiffing up evaluation reports and presentations.

Cool Tricks:

1. Icons: Icons are a fantastic way to communicate ideas without words, and there are some wonderful websites out there from which to download icons. But, did you know that Office 2016 has its own icons? Simply go to the Insert tab, and you will find the Icons option between Shapes and SmartArt.

The downside is that there are far fewer options than, say, Noun Project, but the great part is that you can easily change the color of the icon (to any color!). Just click on the icon, go to the Format tab, and select a color from the Graphics Fill menu. Hats off to you, Microsoft!

2. Maps: You may have noticed that Office 2016 has some new chart types, one of which is a Map chart. My colleague showed me just how easy it is to create a map with this feature. She had a column of county names and a column of numbers, and simply by clicking Insert > Map, a map of that state was produced and the data was mapped and scaled accordingly. See an example of a default map below with some sample data.

You can plot country/region, state/province, county, or postal code, on these maps but not cities or street addresses. The color scale can be a sequential 2-color scale (as shown) or a diverging 3-color scale. To create the map, Microsoft sends your data to Bing Maps, which may or may not be an issue if you are mapping sensitive data.

3. Merge shapes: I recently stumbled across the Merge Shapes command in PowerPoint, which might not be a new feature, but it was new to me. I learned that it can do some fun things when merging a picture with words for added impact. Check it out:

To access the Merge Shapes command, select at least two shapes, images, or text boxes that you want to merge. From the Drawing Tools tab, select the Merge Shapes drop-down menu and choose one of the five commands shown below.

For a description of each Merge Shape command, see this Indezine article.

We are excited to keep exploring Office 2016 to see what other new features or not-yet-discovered features it has!

The American Evaluation Association is celebrating Data Visualization and Reporting (DVR) Week with our colleagues in the DVR Topical Interest Group. The contributions all this week to aea365 come from DVR TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Hi! I’m Kate Diaz, Senior Manager for Corporate Measurement, TechnoServe. When working with data visualizations, it can sometimes be difficult to craft the message that drives home the key insight for your audience. That was an issue we faced as we began to plan for our 2016 Impact Report, which uses data visualizations to help readers understand key aspects of TechnoServe’s impact. Thanks in no small part to the  expert guidance from AEA’s Ann Emery, however, we finished with a report that makes a strong, clear argument for our impact.

Lesson Learned: While creating the report’s data visualizations, I learned how much the revisions process helps refine and isolate the message. When drafting a visualization, I often start out with three or four insights from the data that I want to convey to readers. Through revisions, by sharing the visualization with others, and putting it aside and coming back to it, I hone in on the exact message that drives the report’s narrative. The revisions process helps identify the message just as much as it helps identify the right visual.

Revisions included sketches on pen and paper, the app Paper54, Excel, and photos taken during Skype calls (Thanks again, Ann!)

For example, a key visualization in the 2016 Impact Report illustrates how we assess Financial Benefits, a measure of increased revenue and wages. Early iterations, shown above left, explored the idea of impact flows over time. But we were really interested in talking about 2016 impact, not a trendline. We tried a waterfall chart, which showed the composition of the whole but still accentuated a chronology. Doing away with a timeline, we tried stacked bar charts. These ultimately helped us identify the key insight for readers: that this year’s impact is a result of work we’re doing this year and the sustained impact from prior years. A few iterations later, our message was clear:

The finished product.

After so much effort, it can be hard to let go of the other hard-won insights about our impact in order to isolate the one that best fits the narrative. I’m often tempted to try to convey three or four different insights in one graph. But the result is a busy, messy visualization that is unsuccessful at driving any message home. TechnoServe’s annual impact report is a printed document, so space is limited, but online or presentation versions are more flexible. They are an opportunity to explore the insights that didn’t make it into the printed report.

Data visualization is certainly about the destination: it’s important to land on the graph that clearly conveys the intended message. But the journey, the revisions process, will ensure you get there. Happy graphing!

The American Evaluation Association is celebrating Data Visualization and Reporting (DVR) Week with our colleagues in the DVR Topical Interest Group. The contributions all this week to aea365 come from DVR TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

No tags

I’m Giovanni Dazzo, co-chair of the Democracy & Governance TIG and an evaluator with the Department of State’s Bureau of Democracy, Human Rights and Labor (DRL). I’m going to share how we collaborated with our grantees to develop a set of common measures—around policy advocacy, training and service delivery outcomes—that would be meaningful to them, as program implementers, and DRL, as the donor.

During an annual meeting with about 80 of our grantees, we wanted to learn what they were interested in measuring, so we hosted an interactive session using the graffiti-carousel strategy highlighted in King and Stevahn’s Interactive Evaluation Practice. First, we asked grantees to form groups based on program themes. After, each group was handed a flipchart sheet listing one measure, and they had a few minutes to judge the value and utility of it. This was repeated until each group posted thoughts on eight measures. In the end, this rapid feedback session generated hundreds of pieces of data.

Hot Tips:

  • Add data layers. Groups were given different colored post-it notes, representing program themes. Through this color-coding, we were able to note the types of comments from each group.
  • Involve grantees in qualitative coding. After the graffiti-carousel, grantees coded data by grouping post-its and making notes. This allowed us to better understand their priorities, before we coded data in the office.
  • Create ‘digital flipcharts’. Each post-it note became one cell in Excel. These digital flipcharts were then coded by content (text) and program theme (color). Here’s a handy Excel macro to compute data by color.

  • Data visualization encourages dialogue. We created Sankey diagrams using Google Charts, and shared these during feedback sessions. The diagrams illustrated where comments originated (program theme / color) and where they led (issue with indicator / text).

Lessons Learned:

  • Ground evaluation in program principles. Democracy and human rights organizations value inclusion, dialogue and deliberation, and these criteria are the underpinnings of House and Howe’s work on deliberative democratic evaluation. We’ve found it helpful to ground our evaluation processes in the principles that shape DRL’s programs.
  • Time for mutual learning. It’s been helpful to learn more about grantees’ evaluation expectations and to share our information needs as the donor. After our graffiti-carousel session, this entire process took five months, consisting of several feedback sessions. During this time, we assured grantees that these measures were just one tool and we discussed other useful methods. While regular communication created buy-in, we’re also testing these measures over the next year to allow for sufficient feedback.
  • And last… don’t forget the tape. Before packing your flipchart sheets, tape the post-it notes. You’ll keep more of your data that way.

The American Evaluation Association is celebrating Democracy & Governance TIG Week with our colleagues in the Democracy & Governance Topical Interest Group. The contributions all this week to aea365 come from our DG TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Good day, I’m Bernadette Wright, program evaluator with Meaningful Evidence, LLC. Conducting interviews as part of a program evaluation is a great way better understand the specific situation from stakeholders’ perspectives. Online, interactive maps are a useful technique for presenting findings from that qualitative data to inform action for organization leaders who are working to improve and sustain their programs.

Rad Resource: KUMU is free to use to create public maps. A paid plan is required to create private projects (visible only to you and your team).

Here are the basic steps for using KUMU to integrate and visualize findings from stakeholder conversations.

1) Identify concepts and causal relationships from interviews.

Using the transcripts, you focus on the causal relationships. In the example below, we see “housing services helps people to move from homelessness to housing” (underlined).

2) Diagram concepts and causal relationships, to form a map.

Next, diagram the causal relationships you identified in step one. Each specific thing that is important becomes a “bubble” on the map. We might also call them “concepts,” “elements,” “nodes,” or “variables.”

Lessons Learned:

  • Make each concept (bubble) a noun.
  • Keep names of bubbles short.

 

3) Add details in the descriptions for each bubble and arrow.

When you open your map in KUMU, you can click any bubble or arrow to see the item’s “description” on the left (see picture below). Edit the description to add details such as example quotes.

4) Apply “Decorations” to highlight key information.

You can add “decorations” to bubbles (elements) and arrows (connections) using the editor to the right of your map. For the example map below, bigger bubbles show concepts that people mentioned in more interviews.

Also, green bubbles show project goals, such as the goal “People transitioned out of homelessness.”

Cool Tricks:

  • Create “Views” to focus on what’s most relevant to each stakeholder group. To make a large map manageable, create and save different “views” to focus on sections of the map, such as views by population served, views by organization, or views by sub-topic.
  • Create “Presentations” to walk users through your map. Use KUMU’s presentation feature to create a presentation to share key insights from your map with broad audiences.

Rad Resources:

  • KUMU docs. While KUMU takes time and practice to master, KUMU’s online doc pages contain a wealth of information to get you started.
  • Example maps. Scroll down the KUMU Community Page for links to the 20 most visited projects to get inspiration for formatting your map.
  • KUMU videos. Gene Bellinger has created a series of videos about using KUMU, available on YouTube here.

Organizations we work with have found these map presentations helpful for understanding and the situation and planning collaborative action. We hope they are useful for your evaluation projects!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Hi! I’m Sarah Dunifon, Research and Evaluation Associate at the Wildlife Conservation Society (WCS). In my role, I create many reports and I’m always looking for efficient tools for data visualization. I’ve found a few different programs to display location data, which I’d like to share with the wider AEA community.

Rad Resources:

Google Fusion Tables

Google Fusion Tables is an experimental app add-on that you can link to Google Drive. It allows users to create online, interactive heat maps and feature maps. Privacy settings are managed in the same way as other Google products where users can decide on a range between public and only available to you as the user. Viewers can manipulate the maps in various ways, such as filtering results, scrolling around the map view, or switching between map types.

Feature map of countries with WCS offices – Fusion Tables

Infogr.am

Infogr.am offers quick and easy interactive heat maps which can be shared via weblink. The free version includes a United States map and a world map, and your data will be public, whereas the paid versions have data privacy, more map choices, and the option to download the maps as images.

Heat map of countries with WCS offices – Infogr.am

Excel Apps – Geographic Heat Map

With the “Geographic Heat Map” app on Microsoft Excel, you can create either a world or United States heat map. The data is private and you can save your final map as a picture, making it a good option for inserting into a report. This app doesn’t have much customizability in color and style, but I’ve been able to paste the image into another program (say Microsoft Word or PowerPoint) and edit the image there.

Heat map of countries with WCS offices – Geographic Heat Map with Excel Apps

Tableau

Tableau offers feature maps and heat maps for free, though the data will be public. This program is highly customizable and makes some beautiful visualizations. However, you might find there is a bit of a learning curve to using this software. The visualizations can be saved as an interactive display in “presentation mode” or uploaded to the Tableau Public gallery where they can be shared digitally.

Heat map of countries with WCS offices – Tableau

Excel Powermap

Powermap in Microsoft Excel lets you create private feature maps in a variety of themes on a 3D globe. The map can be an online interactive display or an image produced by taking a screen grab through the program. The screengrab puts a picture of the image onto clipboard, which you can then paste into another program.

Feature map of countries with WCS offices – Excel Powermap

Hot Tip: Consider how you intend to use the map before you start building it. If it needs to be interactive, choose an online format. If it needs to be put into a report, pick a program with capabilities to export a high-resolution image, rather than just a screenshot.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

·

Greetings! I am Kendra Lewis, Evaluation Coordinator for the California 4-H Youth Development Program at University of California, Agriculture and Natural Resources. Today I am going to share my experience with a “data party” as a way to engage stakeholders in evaluation data. We recently held a one-day workshop with 4-H camp staff (youth and adults) to review evaluation data collected at their camps last summer. We had nearly 30 people representing 6 camps attend. We presented results from across the state as a whole as well as specific results for each camp. Evaluation data was collected from two sources: youth campers and teen camp staff. I presented data in multiple representations (graphs, tables, word clouds) and posed open-ended prompts to initiate conversation.

Lessons Learned: Participants loved having the opportunity to explore the data, discuss what they thought the results meant, and formulate action plans with their camp team. The data party made the results accessible and understandable. All camps signed up to participate in the evaluation again, and we already have our next data party planned for Fall 2017 after this summer’s camps.

Hot Tip: Start with a “gallery walk” that gives an overview of the results. We had large posters that presented data from the state results, and had small groups of people walk around to review the posters. We made sure to mix youth with adults, and to put people from different camps together to ensure diversity in camp experiences.

Hot Tip: Create a “data placemat” for each site. We made a data placemat that was specific to each camp that they could review with their team. We made a placement for camper data and a placemat for the teen data so that the data could be reviewed separately for those different experiences.

Hot Tip: Word clouds are a great way to introduce qualitative data. Before giving attendees all the qualitative data, we presented word clouds so as not to overwhelm them. After reviewing the word clouds, each camp had the opportunity to go over all their qualitative data in full.

Rad Resource: Check out all these great ideas and pins from Kylie Hutchinson on data parties.

Rad Resource: See the Innovation Network’s slide deck on Data Placemats for more information about this cool tool.

Gallery Walk

Gallery Walk

Reviewing Data Placemats

Reviewing Data Placemats

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

My name is Dr. Michelle Chandrasekhar and I serve as Board Secretary for the Southeast Evaluation Association (SEA).  My work experience includes higher education and state government, and recently with local, state, and federal criminal justice agencies. Working in different venues reminded me that our evaluation reports share several key elements across disciplines, audiences, and purposes. Below are two of these common elements.

  • What we produce must be faultless. In talking about her report strategies used at the S. General Accounting Office’s Program Evaluation and Methodology Division, Eleanor Chelimsky told a 2006 AEA Conference audience that the reports her office produced had to be accurate. If there was any kind of error, it could provide justification for ignoring or refuting the report.

Hot Tip: Hard to read reports are not used. Carefully proofread your writing, logic, and results. Use a checklist and get multiple people to review the document. Ask for examples of previous reports the clients have liked or hated to review and reference for developing future reports.

  • The audience that reads your report has a different agenda from yours. Chelimsky also said that politicians (and we can agree, any decision-maker) understand evaluation within the context of their own agendas. Evaluators need to be aware of those agendas and skilled at presenting a credible case for their work.

Hot Tip: Reports tell a story and should be written bearing in mind the interests of your audience and what they do and do not know. Tell your audiences about The Characters (Who asked for this report? Who is involved?), The Setting (Why was this report requested? Why was the data collected?), The Plot (What are the research questions? What is the study design?), The Conflict (What are the issues or caveats?), and The Resolution (What are the results and recommendations?). Yes, even an internal report can include recommendations – you know the data!

Rad Resources: Check out these links for further reading:

The American Evaluation Association is celebrating Southeast Evaluation Association (SEA) Affiliate Week with our colleagues in the SEA Affiliate. The contributions all this week to aea365 come from SEA Affiliate members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

<< Latest posts

Older posts >>

Archives

To top