Judy Savageau and Linda Cabral on Using Journal Clubs to Promote Best Practices in Evaluation

Hello! We’re Judy Savageau and Linda Cabral from the Center for Health Policy and Research at the University of Massachusetts Medical School. Recently, we developed a journal club which convenes interested evaluators quarterly to discuss selected article(s) on evaluation research principles or methodologies. The journal club also networks parties from across our medical school’s multiple departments, centers, and campuses that have an interest in evaluation. For each one-hour session, a facilitator guides the discussion based on “questions to consider” which are developed and distributed with the selected reading(s) ahead of time. Some examples of the articles we’ve discussed include:

• Skolits GJ, Morrow JA and Burr EM. Reconceptualizing Evaluator Roles. American Journal of Evaluation 2009;30(3):275-295.

• Smith NL. An Analysis of Ethical Challenges in Evaluation. American Journal of Evaluation 2002:23(2):199-206.

• Morris M. The Good, the Bad and the Evaluator: 25 Years of AJE Ethics. American Journal of Evaluation 2011;32(1):134-151.

• Cohen DJ and Crabtree BF. Evaluative Criteria for Qualitative Research in Health Care: Controversies and Recommendations. Annals of Family Medicine 2008;6(4):331-339.

Skolits, Morrow and Burr’s article, one of the AJE’s most popular articles of 2010, generated some interesting discussion about the different roles which journal club members have assumed as evaluators along with the benefits and challenges those roles entailed. We ask participants to consider the relevance and applicability of the selected reading to their work, then end each journal club with a short debriefing session, taking suggestions for articles and recruiting facilitators for future journal club meetings.

Hot Tips:

• When coordinating with people from different sites, schedule the journal club toward the end of the work day so that participants don’t have to return to their office.
• Alternate the location of the journal club meetings among participants’ sites/campuses to share the travel burden as well as be able to visit new sites and meet potential new collaborators.
• Provide light refreshments to maintain an informal atmosphere for lively discussion.
• Keep the group size fairly small (10-12 people) to ensure active participation among group members.

Rad Resource:

• Use Doodle (http://doodle.com) to identify the best dates/times for people to meet.

Lesson Learned: While there can be challenges in bringing together individuals from different sites, these are balanced by the benefits of getting to know the work of others and learning about the methodologies and strategies they’ve used with varying projects, clients, stakeholders and funding sources. It can also offer opportunities for identifying new evaluation projects to work on together.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.