My name is Amy A. Germuth, President of EvalWorks, LLC (http://EvalWorks.com) and owner/blogger at EvalThoughts.com. I’ve worked over the last year on improving my evaluation reports to better meet my client’s needs and have a few great resources to help you do the same.
Rad Resource: “Unlearning Some of our Social Scientist Habits” by Jane Davidson (independent consultant and evaluator extraordinaire, as well as AEA member and TIG leader). http://davidsonconsulting.co.nz/index_files/pubs.htm She recently added some additional thoughts to this work and presented them at AEA’s 2009 annual conference in Orlando. Her PowerPoint slides for this presentation can be found at: http://bit.ly/7RcDso.
Frankly, I think this great article has been overlooked for its valuable contributions. Among other great advice for evaluators (including models or theories but not using them evaluatively and leaping to measurement too quickly), she addresses these common pitfalls when reporting evaluation findings: (1) Not answering (and in some cases not even identifying!) the evaluation questions that guided the methodology, (2) reporting results separately by data type or source, and (3) ordering evaluation report sections like a Master’s thesis. This entertaining article and additional PowerPoint slides really make a case for using the questions that guide the evaluation to guide the report as well.
Rad Resource: The “Evaluation Report Checklist” by Gary Miron (professor at Western Michigan University and former Chief of Staff at The Evaluation Center at WMU) provides a great outline of the eight main sections in an evaluation report (Title page, Exec. Summary, Table of Contents, Introduction and Background, Methodology, Results, Summary and Conclusion, References) and the various things that should be included in each. http://www.wmich.edu/evalctr/checklists/checklistmenu.htm
The author notes that this checklist can be used as a “tool to guide a discussion between evaluators and their clients regarding the preferred contents of evaluation reports and a tool to provide formative feedback to report writers” and can help writers identify the strengths and weaknesses of their report. However, as Gary notes, evaluation reports differ greatly in terms of purpose, budget, expectations, and needs of the client, thus one may need to consider or weight the checkpoints within sections as well as the relative importance and value of each section when reviewing one’s own writing (or someone else’s).
Using the Evaluation Report Checklist in conjunction with some of Dr. Davidson’s suggestions has increased the quality and utility of my evaluation reports and should do the same for yours.
This contribution is from the aea365 Daily Tips blog, by and for evaluations, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org
For members only, the screencast and recording of the Coffee Break Webinar Amy Germuth offered on Using the Fantastic Five Checklist may be accessed here http://bit.ly/GermuthCheckweb.
Not a member? Her handout on resources for learning more about the Fantastic Five Checklist may be found in the AEA Public eLibrary here http://bit.ly/GermuthCheckhand. I encourage you to consider joining and thus gaining access to AEA’s webinars archive library (as well as journals, professional development, thought leaders discussions, newsletters…). Join now online at http://www.eval.org/membership.asp.
Pingback: Web Perspectives: Amy A. Germuth on Evaluation Blogging « Fresh Spectrum
And, one more item, Amy is the facilitator for a demonstration as part of AEA’s Coffee Break Demonstration Series. She’ll be showing how to use the Fantastic Five Checklist to Write Better Survey Questions and Improve Survey Reliability, January 12 at 2:00 pm EST. Free for AEA members (others may purchase a demonstration series pass that includes free membership!). More info may be found at http://www.eval.org/demos.asp
Thanks for sharing Amy – and for being brave enough to be the very first contributor to the aea365 blog.
I wanted to note that Jane Davidson will also be the discussant for the AEA thought leaders series the week of January 17 for those who want to connect directly with her.
http://www.eval.org/thought_leaders.asp
Each month, we host an asynchronous online exchange over the course of a week using AEA’s discussion forums. The thought leaders series is free for AEA members!
If others have suggestions for resources for improving reporting, please feel free to share via the comments.