We are Chris Coryn and Lyssa Wilson from the Interdisciplinary Ph.D. in Evaluation program at Western Michigan University. In the last decade, research on evaluation theories, methods, and practices has increased considerably. Even so, little is known about how frequently published findings from research on evaluation are read and whether such findings influence evaluators’ thinking about evaluation or their evaluation practice. To address these questions, and others, we (including our colleagues Satoshi Ozeki, Gregory Greenman II, Daniela Schröter, Kristin Hobson, Tarek Azzam, and Anne Vo) recently completed a study using a random sample of AEA members and a purposive sample of prominent evaluation theorists and scholars.
Lessons Learned:
- Nearly all (96.94% ±38%) AEA members and all (100%) theorists and scholars consider research on evaluation important
- A majority of AEA members (80.95% ±60%) and theorists and scholars (84.21%) regularly read research on evaluation
- A majority of those sampled indicate that research on evaluation has influenced their thinking about evaluation and their evaluation practice (97.00% ±38% and 94.00% ±4.79% [for AEA members] and 100% and 100% [for prominent theorists and scholars], respectively)
- The American Journal of Evaluation and New Directions for Evaluation are, overall, the most frequently read journals by a majority of AEA members (70.35% ±76% and 51.18% ±7.44%, respectively)
- In addition to the American Journal of Evaluation and New Directions for Evaluation, prominent theorists and scholars tend to also read other journals semi-regularly or regularly (e.g., Evaluation: The International Journal of Theory, Research and Practice, Journal of MultiDisciplinary Evaluation)
- AEA members most often read articles on evaluation methods (92.85% ±64%), reflections on evaluation practice (87.80% ±6.15%), or research on evaluation (80.95% ±7.60%), whereas theorists and scholars most often read articles on evaluation theory (94.73%), evaluation methods (89.47%), research on evaluation (84.21%), and evaluation ethics (84.21%)
- For AEA members, research on evaluation has significantly influenced their thinking about evaluation and their evaluation practice (97.00% ±38% and 94.00% ±4.79%, respectively)
- Research on evaluation has influenced all theorists and scholars’ thinking about evaluation as well as their evaluation practice (100% and 100%, respectively)
- AEA members and prominent theorists and scholars believe that findings from research on evaluation contribute to ‘improving, informing, and guiding evaluation practice’ (40.59% and 50.00%, respectively)
Rad Resources:
Christie’s article ‘Advancing empirical scholarship to further develop evaluation theory and practice’ in the Canadian Journal of Program Evaluation (2011)
Henry and Mark’s article ‘Toward an agenda for research on evaluation’ in New Directions for Evaluation (2003)
Szanyi, Azzam, and Galen’s article ‘Research on evaluation: A needs assessment’ in the Canadian Journal of Program Evaluation (2012)
The American Evaluation Association is celebrating Research on Evaluation (ROE) Topical Interest Group Week. The contributions all this week to aea365 come from our ROE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.