Greetings current or future journal reviewers! My name is Shannon Hitchcock, PhD (she/her/hers) and I am a quantitative sociologist, public health evaluator, and, since February 2023, the Managing Editor of the American Journal of Evaluation. I am here to, hopefully, encourage you to become an active reviewer for AJE, if not already! The content of this blog post is based on a presentation given by AJE Co-Editor, Dr. Laura Peck, and me this past October at the annual American Evaluation Association conference held in Indianapolis, IN.
Why (and Why Not) to Review
Journal editors are not all-knowing, we need input from experts, like yourself, to inform the review process. Reviewing is good for you! It is often seen as a professional responsibility, you get to learn about the latest research, and it builds your reputation in the field. Importantly, you can say no! In fact, we encourage you to do so if it’s a bad fit or if you simply do not have the bandwidth. We always welcome reviewer suggestions. Send your astute colleagues our way, and we’ll bug you in the future!
How to Become a Journal Reviewer
When inviting reviewers, it’s common for editors to lean on their networks. Editors know you from the field, your work, and your reputation. It’s also common for editors to source the manuscript’s reference list for those publishing related work. Editors also appreciate reviewer recommendations. However, these common approaches for inviting journal reviewers are far from equitable and may create a barrier to entry for new, interested, and more than capable reviewers. AJE is committed to a more equitable approach to inviting reviewers as we work with our new Editorial Team and Editorial Advisory Board. We are open to your suggestions on how to do better (a few recent, and great, suggestions – volunteer reviewers, junior reviewers, mentorship). Please check out Hot Tip (below) for our current initiative.
How to Be a (Great!) Journal Reviewer
First and foremost, “be kind” (Lee, 1995). Below is a list of “action”-able steps for how to be an effective journal reviewer adapted from Dr. Allen S. Lee’s (1995), “Reviewing a Manuscript for Publication.”
- Pay attention to deadlines.
- Briefly summarize the manuscript in your own words, what’s the big picture, identify strengths – this will allow the author to assess if their intended purpose is actually coming across.
- Explain your positionality – what you bring (and don’t bring) to the review (I can comment on “[topic],” but probably not “[methods]” (or vice versa).
- Organize your review into:
- Big points (or deal breakers) (e.g., conceptual framework, measures, design, analysis, interpretation)
- Small points (e.g., excessive use of passive voice, typos)
- Include page/location references for your comments.
- Identify missed sources, if appropriate (and provide references).
- Thank the author for the opportunity to read their work.
Lessons Learned
A favorable review may still be “bad” if it’s vague or offers little direction. Yes, most authors will gladly welcome a favorable review, regardless if it checks off all the “action”-able steps, but we also want to know how to improve our work (or if there’s a better fit at a different journal). A good review is useful in assessing the manuscript, including its weaknesses and strengths.
Hot Tip
If you’re on board, please create/update your account’s contact information and areas of focus on our ScholarOne ManuscriptsTM webpage, and then complete this Google Form to join our shortlist of volunteer reviewers.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.