AEA365 | A Tip-a-Day by and for Evaluators

TAG | presentations

Hello, we are Greg Lestikow, CEO and Fatima Frank, Project Manager of evalû, a small consulting firm that focuses exclusively on rigorous evaluations of social and economic development initiatives.  We champion impact evaluation that maintains academic rigor but is based entirely on our clients’ need to improve strategic and operational effectiveness and increase profitability.

We spend much of our time in the field, yet we often find that we have less time on the ground than we’d like. We have a lot to do and a lot of information to transfer to our clients’ field teams. Just as importantly, we are trying to build local buy-in to ensure a successful evaluation once we leave the country.  As such, we have developed and refined three basic workshops to involve key local staff early on in the evaluation process.

Hot Tips:

  • In the first week, cover the basics of evaluation. Given the diversity of field staff and skill sets evaluators work with, it’s important to lay out the basic groundwork of M&E so that everyone is on the same page when it comes to the purpose of the evaluation. Our first presentation, titled “Measuring Impact,” covers:
  • Distinguishing between monitoring and evaluation
  • Discussing the importance of rigorous evaluation and the project evaluation process
  • Introducing methodologies for rigorous evaluation techniques
  • Drawing links between program design and program evaluation
  • Understanding how we measure change at the individual and community level
  • Second, (generally during the same week), give an in-depth presentation on evaluation indicators. This would be an ideal time to present an Indicator Framework if you have one (see our three-part series on our evalû-created indicators system) or give a general presentation on indicators (what makes a good indicator, SMART indicators, etc).  This presentation should provide practical training with participatory activities to engage the field teams and ensure that they understand the nuances and organization of indicators and requirements. This training helps M&E and program staff think about their projects and what indicators they want to use for the evaluation.
  • The last presentation should occur in the week or two before start of data collection. This presentation consists of specific training for survey enumerators and qualitative data collection workers.  We combine the training for surveys and qualitative data collection so that workers in both areas understand how the instruments support one another.  We’ve also found that this cross-training allows data collectors to take on different responsibilities if needed.  You never know when one of your enumerators will call in sick!

If you have other helpful evaluation training resources, please feel free to post them here or contact us directly.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Greetings, I am June Gothberg and assisting with AEA’s Potent Presentations Initiative (p2i). Today, I want to share with you how to design presentations for all audience members. It is important that presentations are readable, navigable, and understandable.

Lesson Learned: Your audience is likely to be diverse.

  • The World Health Organization (WHO) estimates:
    • 7-8% of men and 0.4% of women are colorblind, across eight different strains (red/green, blue/yellow, full colorblindness).
    • 4% are low vision.
    • 4% are severely dyslexic.
    • Many are slow readers or processors.

Hot Tips:

  • Emphasize readability: Fonts should be large and easy to read.
    • Use Sans Serif fonts such as Helvetica, Arial and Verdana rather than font types like ‘Times New Roman’, because low vision people have difficulty with reading text in font types with serifs.
    • Avoid the use of italic font style; this style even hampers normal vision reading.
    • Try to use one font type per slide.
  • Emphasize see-ability: Color selection is important.
    • Use solid or near solid backgrounds.
    • Color contrast can improve see-ability.
    • There are two types of contrast – brightness and color.
    • The highest brightness contrast, thus increased see-ability, is black and white.
    • Use complementary colors (colors directly across from each other on the color wheel).
    • Red-green is the highest incidence of colorblindness, so use these colors sparingly.
  • Emphasize understandability: Graphics, figures, maps, and images are oft neglected.
    • Use oral support when presenting graphics.
    • Remember red is a color issue – laser points are red!
    • Legends are difficult for many to interpret.
    • Use text, arrows, or other indicators to emphasize important information.
  • Handouts may be critical especially for those with hidden disabilities.
    • Distribute handouts before you present; this especially assists those with low vision, dyslexia, and comprehension difficulties.
    • Bring an electronic copy.
    • If possible, bring a few large-text copies.
  • Attend Potent Presentations Initiative trainings on slide design at the conference.

Rad Resources:

  • To check overall accessibility. Instructions for using Microsoft’s Accessibility Checker
  • To check your design: Try Petr Stanicek’s Color Scheme Designer. Select your intended color scheme and then select the colorblindness button in the top right corner to check your scheme for each of the eight strains of colorblindness.
  • To check your slides: Try Vischeck. Once you create your presentation, create a picture image of your slide(s). This is very simple to do by using screen capture (PC and Mac instructions), open up your graphics editor, crop, and save as a .png or .jpg file. Next upload to Vischeck and viola you can now see your slide as someone else might.
  • P2i Presentation Slide Design Guidelines.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Associationand provides a Tip-a-Day by and for evaluators.

· ·

I’m Susan Kistler, the American Evaluation Association’s Executive Director and aea365 Saturday contributor. Today I want to talk about one of the hot debates in presentation delivery: To Prezi or not to Prezi, that is the question.

For the uninitiated, Prezi is a presentation tool that is a possible replacement for PowerPoint. It allows the presenter to zoom in and out of various points on a larger graphic. Penny Black wrote about Prezi in a very popular earlier aea365 post that is worth reading in order to gain an understanding of its functionality. Stephanie Evergreen, projector director for AEA’s Potent Presentations Initiative (and substitute for me last week on aea365 – thanks Stephanie), recently wrote a blog post on “Why I’m Not in Love with Prezi” on her blog, and I chimed in about my own dislike for the tool. Then, at the AEA/CDC Summer Evaluation Institute, at a workshop session on low cost tech tools, we discussed when Prezi might be exactly what the doctor ordered.

Rad Resource: It is difficult to explain what Prezi is without taking a look. If you haven’t seen Prezi in action, you need to see an example to understand. Colleagues from New Zealand presented at Evaluation 2010 using Prezi and their presentation may be found online here.

Lessons Learned: My dislike of Prezi stems from a two key issues:

  • It can literally make the audience queazy watching a Prezi presentation
  • Using Prezi requires a level of design skill that is rarely in evidence on the part of the presentation creator – to make a great Prezi you need to attend to both the whole and the parts

Yet, it is a new platform, and platforms evolve over time. It is a tool, and like any tool, it can be wielded well or poorly. Our workshop discussion tried to consider the strengths, and two came to mind:

  • It adds novelty to a presentation – a bit of uniqueness. Evaluation presentations can be stodgy and benefit from tools that capture and hold attention. Although we discussed the downside, in that people may be attending to the process rather than the presentation content, this was still felt to be a strength.
  • In the hands of a good designer, it likely can illustrate well parts of a whole, moving back and forth between the two. Look at the New Zealand example, about half way through there is a lovely diagram of a tree and the Prezi presentation moves us in and out from its components to the full illustration.

Get Involved: Have you tried Prezi? What works for you? I am, in particular, in search of a Prezi presentation of a logic model – moving between the parts and whole to good effect. Do share!

The above is fully my own opinion and does not reflect that of the American Evaluation Association.

· ·

Hey there. I’m Stephanie Evergreen, AEA’s eLearning Initiatives Director and general data communications geek. Susan Kistler has a family obligation this weekend, so I’m stepping in to share with you the newest developments in AEA’s Potent Presentations Initiative (p2i).

Potent Presentations Logo

You’ve heard about p2i, right? It is a new initiative to help AEA members improve their presentation skills, particularly around delivering conference presentations. We come together once or twice a year to teach each other about our practices and processes, so shouldn’t we do everything we can to make it easy to learn from our presentations? That’s why p2i will feature online and in-person training before and during the annual conference around the three facets of presenting: message, design, and delivery.

We have just launched p2i.eval.org, which will be the hub of this activity.

Rad Resource: Our home page features our upcoming webinar-based training on how to prepare for and deliver an Ignite session. When you receive the proposal status notice for your Ignite session on July 3, head to our site to sign up for one of the two trainings, either on July 17 at 11:30am ET or July 26 at 4pm ET.

Rad Resource: Our first tool to help you rock your conference session is the Presentation Preparation Checklist. Download this PDF to find out what to prepare when, keep yourself on track, and minimize the last minute rush many people experience leading up to a conference presentation. The checklist include time frames specific to this year’s annual conference, October 22-28.

Rad Resource: During the conference we’ll provide a demonstration on research-based effective practices around slide design. But you don’t want to wait until then to begin working on your session slides. So we’ve released the handout for that demonstration already. Head to the p2i site to snag the Slide Design Guidelines (with extra tips for handouts, too). It covers how to handle fonts, graphics, colors, and arrangement and includes links for step-by-step instructions (we’ll add links each month) and awesome extensions of these guidelines from your AEA colleagues.

· · · · · ·

I’m Susan Kistler, AEA’s Executive Director, and regular aea365 Saturday contributor. I was inspired by Dorian’s 4/28 post on 5 must-read blogs for those working in nonprofits, and thought I would share a few must see/hear webcasts and podcasts with which you may not be familiar.

Rad Resource – Videos on Evaluation Methods from UNICEF: Recorded during a 2006 professional development training session, each video may be viewed in your browser and is over an hour in length. The evaluation topics include from AEA members include:

  • Jim Rugh on Real World Evaluation
  • David Fetterman on Empowerment Evaluation
  • Huey Chen on Theory Driven Evaluation
  • Bob Williams on Systems Approach to Evaluation
  • Sanjeev Sridharan on Multilevel Models in Program Evaluation

Rad Resource – Nancy Duarte’s YouTube Channel: Nancy Duarte, author of slide:ology and resonsate, is a vanguard in the post-Tuftean movement to create great presentations. Duarte urges us to tell stories, to make the audience care, to get rid of the extraneous and focus on a central message clearly conveyed. Sound like advice you could use for your AEA, client, or stakeholder presentation? Her short YouTube videos illustrate and expand the concepts from her books.

Rad Resouce – Social Good Podcasts from the Chronicle of Philanthropy: Hosted by AEA Member Allison Fine, a 2010 keynote speaker at the AEA/CDC Summer Evaluation Institute, this podcast focuses on “how charities and foundations can more effectively use social media tools.” While not aimed explicitly at evaluation or measurement, Allison’s background as the founder of Innovation Network, a leading evaluation consulting firm, is part of the lens she brings to her interviews and discussions with leaders in the nonprofit sector.

By the way, this year we’re pleased to welcome 2010 AEA President Leslie Cooksy speaking on Quality and the Good Enough Rule, and Zach Gemignani, founder of Juice Analytics, talking about 10 Steps to Data Vizardry, as our keynotes for the 2011 Institute coming June 12-15 to Atlanta. Hope to see you there!

Rad Resource – Filmspotting Podcast: Time for a break? Adam Kempenaar and Matty Robinson review and discuss films each week – always a new release or two (and rarely the blockbusters but something a bit more esoteric) and then a look back into the best from the past. They feed my film lust. I’ve always been intrigued by activities like film critique and wine tasting that hearken back to Eisner’s connoisseurship model of evaluation. They also remind me about the role of judgment and critical examination in our day to day activities – I’ve now come full circle back to last week’s post on thinking evaluatively in your everyday life.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators

· ·

AEA365 began on January 1, 2010. Before we promoted this resource, we reached out to dedicated authors who believed in the project in order to populate the site with starter content. Those who contributed in week 1 wrote for an audience of fewer than 10. One year later we have over 1500 subscribers and are re-posting the contributions from those trailblazers in order to ensure that they receive the readership they deserve for their great ideas – Amy was kind enough to update hers for 2011!

My name is Amy A. Germuth,  and I am Founder and President of EvalWorks, LLC (http://EvalWorks.com) in Durham, NC and blog at EvalThoughts.com. Over the last year I have worked improving my reporting of findings to better meet my client’s needs and have a few great resources to help you do the same.

Rad Resource: “Unlearning Some of our Social Scientist Habits” by Jane Davidson (independent consultant and evaluator extraordinaire, as well as AEA member and TIG leader). She added some additional thoughts to this work and presented them at AEA’s 2009 annual conference in Orlando. Her PowerPoint slides for this presentation can be found at: http://bit.ly/7RcDso.

Frankly, I think this great article has been overlooked for its valuable contributions. Among other great advice for evaluators (including models or theories but not using them evaluatively and leaping to measurement too quickly), she addresses these common pitfalls when reporting evaluation findings: (1) not answering (and in some cases not even identifying!) the evaluation questions that guided the methodology, (2) reporting results separately by data type or source, and (3) ordering evaluation report sections like a Master’s thesis. This entertaining article and the additional PowerPoint slides really make a case for using the questions that guide the evaluation to guide the report as well.

Rad Resource: The “Evaluation Report Checklist” by Gary Miron (professor at Western Michigan University and former Chief of Staff at The Evaluation Center at WMU) provides a great outline of the eight main sections in an evaluation report (Title page, Exec. Summary, Table of Contents, Introduction and Background, Methodology, Results, Summary and Conclusion, References) and the various things that should be included in each.

The author notes that this checklist can be used as a “tool to guide a discussion between evaluators and their clients regarding the preferred contents of evaluation reports and a tool to provide formative feedback to report writers” and can help writers identify the strengths and weaknesses of their report. However, as Gary  notes, evaluation reports differ greatly in terms of purpose, budget, expectations, and needs of the client, thus one may need to consider or weight the checkpoints within sections as well as the relative importance and value of each section when reviewing one’s own writing (or someone else’s).

Rad Resource: Why assume all findings have to be reported as a paper?  Try reporting using PowerPoint and heed the advice Garr Reynold’s provides in his great book “Presentation Zen Design” to ensure that you do not subject your clients to DBP (death by PowerPoint).

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

My name is Steve Fleming, and I am a Senior Systems Analyst for the National Center for Educational Achievement (NCEA) in Austin, TX. NCEA is a department of ACT, Inc., a not-for-profit organization committed to helping people achieve education and workplace success. NCEA builds the capacity of educators and leaders to create educational systems of excellence for all students. We accomplish this by providing research-based solutions and expertise in higher performing schools, school improvement, and best practice research that lead to increased levels of college and career readiness.

NCEA is constantly evaluating how well its data visualizations communicate with educators. We have found books and articles by Stephen Few to be very helpful in this regard. Mr. Few adheres to Edward Tufte’s maxim, “Above all else, highlight the data” in a straightforward exposition of best practices for all types of standard data visualizations.

Rad Resource: Stephen Few’s book, Show me the Numbers, provides an excellent introduction to improving your visual communication skill. A collection of his articles is also available for free at the Perceptual Edge Library.

Hot Tip: Stephen Few provides data visualization recommendations based on how our brains actually work. Did you know…?

  1. The properties of position and color are processed by the visual system first. Emphasizing one of these attributes is often the best way to guide the reader to focus on the most important information. When used inappropriately, these can actually inhibit communication.
  2. Our short term memory can only handle a limited amount of information at a time. For this reason, trying to code more than a few categorical distinctions on a graph is problematic because the reader will forget what each, for example, color stands for.
  3. Our minds fill in gaps in lines and figures. For this reason, it is unnecessary to draw a box around an entire graph. A single vertical and horizontal axis will do fine and reduce chart clutter.

Rad Resource: Do you need to create a data visualization quickly? Juice Analytics Chart Chooser allows you to filter by your charting need to select the data visualization that is right for you. Do you see something you like? Click on the chart, and you have the option to download a template in Excel or Power Point.

This contribution is from the aea365 Tip-a-Day Alerts, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

· · ·

My name is Susan Kistler, the American Evaluation Association’s Executive Director, and I contribute each Saturday’s post to aea365. One hot topic for me this year is data visualization – representing data in ways that are accurate, accessible, and appealing. My very first aea365 post identified resources for those with interests in data visualization – including the classic Periodic Table of Visualization Methods. Today, I want to provide an update.

Hot Tip: Stephanie Evergreen (who gave Great Tips for Graphic Design on July 16) is working to bring together evaluators, and those working in related disciplines, who have an interest in data visualization and reporting in hopes of forming an AEA Topical Interest Group. The group would strive to build capacity and expand the knowledge base in the evaluation field in order to expand stakeholder understanding, improve interpretation, and increase use of evaluation results. As a starting point, Stephanie is building an emailing list of interested individuals and if you would like to be on the list, add a comment to this post (click through back to the website if you received this via email). She’ll also be hosting an informal meeting at Evaluation 2010!

Rad Resources: I attended a presentation in Boston given by the wonderful team at juice analytics as part of their Viva Visualization tour. The presentation was free, definitely worth the 90-minutes of my time, and gave me great ideas for improving my own reports. They’ll be coming to Washington on September 16 if you are in the area. If you aren’t in the DC area, and even if you are, you can learn from their blog – some of the best content can be found on their visitor’s guide and you can subscribe from that page as well. A couple of my favorites? Check out the post on Lightweight data exploration in Excel (under Excel Tricks) to make super-easy inline bars and  Stimulus Bill Explorer (under demos) to see an interactive tree map in action.

Rad Resource: The vizthink group on LinkedIn (you’ll need to join LinkedIn if you aren’t a member but it is free) is a great place to learn from others, post questions, and gather feedback and suggestions.

Hot Tip: Nancy Duarte, author of slide:ology, a book recommended by John Nash in his April 9 aea365 post on Creating Outstanding Presentation Slides. Nancy has a new book coming out – resonate: Present Visual Stories that Transform Audiences due on September 28. She has also created a series of short new videos on creating outstanding presentations, some of which may be found individually on Amazon, but as a set they were uploaded just this week to Vimeo for free viewing.

The above represents my own opinions and not necessarily those of AEA. This contribution is from the aea365 Tip-a-Day Alerts, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

· · ·

I am American Evaluation Association Executive Director Susan Kistler and I contribute each Saturday’s aea365 post. I have a dirty little secret – I’m obsessed with fonts! Stephanie Evergreen recently gave a great AEA Coffee Break Webinar on Graphic Design for Evaluators (she’ll also be expanding on this topic at AEA’s Annual Conference this November). Stephanie expounded on the importance of color choice and font selection to make great reports and presentations.

Rad Resources: WhatFontIs and WhattheFont both allow you to upload a picture of a font or specify a URL and they will return a best guess at the pictured font. WhatFontIs includes the option to display only free or similar free fonts and then download them on the spot – and it has hundreds in its archive (also browsable). Alternatively, WhattheFont also has a forum where font geeks will help you identify a font if you run into a dead end. Here is a great walk-through of from the MakeUseOf blog. Both of these sites are in beta, and aren’t perfect, but I’ve been impressed with the options that they’ve provided – and the sheer volume of free fonts available from WhatFontIs for free.

Stephanie encouraged her webinar attendees to use kuler to identify particular colors appropriate to a report based on those used by a client. Using tools such as WhatFontIs takes this concept one step further.

Rad Resources: This short article from Chuck Green gives examples of sets of fonts that work together to convey a mood or message. I have no eye for such things and find this guidance invaluable.

Hot Tip: Know your vocabulary when talking about fonts with a designer or using fonts in your word processing program. Here are three definitions that will help you along:

  • Serifs: Serifs are the little ‘feet’ that appear on many fonts. Fonts come in two types – Serif fonts such as Times New Roman and Sans serif fonts such as Arial.
  • Kerning: Kerning refers to adjusting the space between letters so that the white space is similar from letter to letter, for instance pushing a ‘A ‘and ‘W’ up close to one another producing ‘AW’ so that they actually overlap in vertical space. You can adjust kerning in Microsoft Word under the character spacing. Kerning is used in particular when creating headlines or banners.
  • Proportional Typefaces: Almost all typefaces today are proportional, allotting varying amounts of horizontal space to a letter based on its shape so that an ‘l’ receives less space than an ‘m’. Old typewriters used monospace fonts.

Rad Resource: A moment of fun for the font obsessed – take a look at this comic that reflects on font choice http://ow.ly/2hf2e.

Note: These insights are my own and do not necessarily represent the opinion of the American Evaluation Association.

· ·

I’m Jim Burdine, Assistant Dean for Public Health Practice and Co-PI/Director of a Prevention Research Center at the School of Rural Public Health, Texas A&M. Over the last 30 years I’ve used community health status assessment as both a community organizing tool and intervention planning tool. In more than 200 different communities (including multiple iterations in the same community) I’ve presented/seen data presented from community assessments in a number of different formats with varying degrees of success and failure.

Lessons learned: What I’ve observed as most important, is presenting the data to them in a manner that matches the expectations of the audience – audiences, really. In other words, the groundwork you’ve already done in your assessment process (hopefully incorporating community-based participatory research principles) should dictate the format. If community members have been involved in planning, conducting and analyzing the data, they should play the major role in presenting the results. If they have been more passively involved, they may expect a “report” FROM you. Obviously the degree of “buy in” to the findings varies dramatically as a function of the degree of participation.

Given that starting place, the next challenges you face are: (1) the sheer volume of information you have to present, and (2) the variation in sophistication around understanding data within a community audience. So first, you have to accept that you can’t present EVERYTHING. You need to decide what are the key points you want to make and focus on those. We’ve all sat through a presentation where somebody reads us the demographics of a community or lists every chronic disease ever found in that population and an hour later you’ve learned nothing new. As a general rule I don’t both to report anything unless it is (1) statistically significantly different from some external reference point (e.g., Healthy People 2010, a state or national rate) and (2) unless there is something that could likely be done locally to impact that problem (it’s actionable).

If well-planned, you will have representatives from all community sectors in your audience (e.g., health care, business, elected officials, religion, education, the media, consumers and representatives of special interests/special needs groups). So you have to decide on what common denominators (e.g., educational attainment, exposure to health statistics) you are going to assume for your audience. You need to be comfortable with knowing that some aren’t going to understand everything you say and other are going to be bored with your “simplistic” presentation. Don’t make the mistake of trying to explain every point to each group n your audience. It just frustrates them and makes for a very ineffective presentation. Plan to do multiple presentations for different audiences rather than a “one size fits all” presentation.

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

· ·

<< Latest posts

Older posts >>

Archives

To top