AEA365 | A Tip-a-Day by and for Evaluators

TAG | evaluation reports

My name is Dr. Michelle Chandrasekhar and I serve as Board Secretary for the Southeast Evaluation Association (SEA).  My work experience includes higher education and state government, and recently with local, state, and federal criminal justice agencies. Working in different venues reminded me that our evaluation reports share several key elements across disciplines, audiences, and purposes. Below are two of these common elements.

  • What we produce must be faultless. In talking about her report strategies used at the S. General Accounting Office’s Program Evaluation and Methodology Division, Eleanor Chelimsky told a 2006 AEA Conference audience that the reports her office produced had to be accurate. If there was any kind of error, it could provide justification for ignoring or refuting the report.

Hot Tip: Hard to read reports are not used. Carefully proofread your writing, logic, and results. Use a checklist and get multiple people to review the document. Ask for examples of previous reports the clients have liked or hated to review and reference for developing future reports.

  • The audience that reads your report has a different agenda from yours. Chelimsky also said that politicians (and we can agree, any decision-maker) understand evaluation within the context of their own agendas. Evaluators need to be aware of those agendas and skilled at presenting a credible case for their work.

Hot Tip: Reports tell a story and should be written bearing in mind the interests of your audience and what they do and do not know. Tell your audiences about The Characters (Who asked for this report? Who is involved?), The Setting (Why was this report requested? Why was the data collected?), The Plot (What are the research questions? What is the study design?), The Conflict (What are the issues or caveats?), and The Resolution (What are the results and recommendations?). Yes, even an internal report can include recommendations – you know the data!

Rad Resources: Check out these links for further reading:

The American Evaluation Association is celebrating Southeast Evaluation Association (SEA) Affiliate Week with our colleagues in the SEA Affiliate. The contributions all this week to aea365 come from SEA Affiliate members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

We are Lauren Baba and Carol Cahill with the Center for Community Health and Evaluation (CCHE), part of Group Health Research Institute (GHRI) in Seattle. Our team of consultants works with various stakeholders to evaluate community health initiatives, clinic-community linkages, and health improvement coalitions. We are always brainstorming new ways to make evaluation reports creative, accessible, and interesting for community partners. Drawing on the team’s 20 years of experience, we have developed reporting recommendations to help evaluators present findings that resonate with different target audiences and clients. In part 1 we focused on report content and in part 2 we consider report formatting.

Hot Tip: Make it easy for your readers to discover the key messages in your evaluation findings. Build in plenty of white space on your pages (which leaves some room for placing key quotations). Take some advice from those who write for the web and keep section headers under seven words and paragraphs under 55 characters wide (8-11 words); see CCHE’s issue briefs for the National Leadership Academy for the Public’s Health.

Hot Tip: Since many reports are read on a screen, consider landscape format. In fact, why not create your reports using PowerPointÔ? You can make a report template with custom colors and fonts, and with creative use of text boxes you can easily create two- or three-column layouts. See Nancy Duarte’s SlidedocsÔ, including templates you can download for free. Landscape orientation also leaves more breathing room for charts and tables. For an example, see this CCHE report on a community program. Be sure and check out Angelina Lopez’s AEA Coffee Break Webinar from April 28 that focused on these alternative reporting formats.

Hot Tip: Speaking of charts and tables, make sure you choose the right chart for your data and consider including a “takeaway message” as the chart title. Simplify formatting of tables to make them clean and easy to read: eliminate extraneous lines, avoid shading, and get rid of duplicate labels.

Rad Resources: Check out this interactive table makeover from Darkhorse Analytics. And remember, both Stephanie Evergreen and Ann Emery provide excellent data visualization advice in their blogs.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

We are Lauren Baba and Carol Cahill with the Center for Community Health and Evaluation (CCHE), part of Group Health Research Institute (GHRI) in Seattle. Our team of consultants works with a various stakeholders to evaluate community health initiatives, clinic-community linkages, and health improvement coalitions. We are always brainstorming new ways to make evaluation reports creative, accessible, and interesting for community partners. Drawing on the team’s 20 years of experience, we have developed reporting recommendations to help evaluators write appropriately for different target audiences and clients.

Hot Tip: Synthesize, don’t just summarize. Clients and community partners are looking for your insights into what evaluation findings mean for their work and communities. Refer to Michael Quinn Patton’s AEA365 tips to help focus your synthesis. Then, captivate audiences early; do not wait to answer the “So what?” question at the end of your report. See CCHE’s executive summary for the Sierra Health Foundation as an example.

Hot Tip: Short reports can pack a bigger punch. Right-size the length and level of detail you share with different audiences. If you write a full-length report, consider pulling together tailored briefs for each target audience’s interests, similar to CCHE’s issue briefs for the National Leadership Academy for the Public’s Health.

Hot Tip: Break the mold and write reports that do not follow the usual order of contents or formatting. Be responsive to your client’s expectations for the report, but try to open their eyes to alternatives:

  • Respect your readers’ time and begin with your key takeaway message, then provide the supporting information.
  • Move methodology sections to the end of your report. Or instead include them in a technical appendix with more detailed discussion and data tables for interested readers.

Rad Resource: Check the reading level of your reports and use plain language guides like GHRI’s Program for Readability In Science & Medicine (PRISM) resources to help you write using language that your audiences will understand.

*Look for Part 2 of our article tomorrow!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

Greetings from the University of Chicago! We are Courtney Heppner and Sarah Rand, Associate Project Directors for Research and Evaluation at the Center for Elementary Mathematics and Science Education (CEMSE) at the University of Chicago. We will be presenting at AEA’s Annual Conference in DC about producing online evaluation reports. Sarah has blogged about online evaluation reports before here. Our team recently completed a new online report on computer science education. You can view that report here.

Lessons Learned:

Online reports allow us to present information in a powerful and dynamic way: Long gone are the days of PDF reports full of tables, charts and lots of text. Online reporting allows us to insert audio and video clips into the report as well as interactive infographics, making the information more engaging for the reader.

Online report development requires the right team of people and is a true test in collaboration: Online reporting requires more than just the evaluation team – it also requires web developers and designers. This team must come together to agree on a process and execute within the set timeline. For more on our work in developing an infographic with a team from Visual.ly, read Sarah’s previous post here.

Online report development requires both time and money: Having more people contribute to the reporting process in turn requires more time and money than it takes to create a traditional PDF report. Our team now builds this cost in to new evaluation budgets.

Rad Resource: One of the best resources we’ve found in our online reporting endeavor has been feedback from our clients. We encourage anyone who decides to develop an online report to follow up with the client to get feedback. Questions we asked the client included:

  • Which part(s) of the report were the most valuable to you?
  • Did you share the online report with anyone? If so, what kind of response did you get about the report?
  • How did you share the report with others? What social media tools did you use?
  • How could we improve our online reporting in the future?
  • Was the online report more useful to your organization than a PDF?

Rad Resource:  Us!  Join us to learn more about online reporting during our session at the AEA Annual conference on Saturday October 19th!

This contribution is from the aea365 Tip-a-Day Alerts, by and for evaluators, from theAmerican Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org. Want to learn more from Courtney and Sarah? They’ll be presenting as part of the Evaluation 2013 Conference Program, October 14-19 in Washington D.C.

Archives

To top