Greetings, AEA365. Liz DiLuzio here, lead curator of the blog. Although today’s post is from the AEA365 archives, it is as relevant now as it was when it was originally posted in 2019. Whether this is your first read or your tenth, I hope you find something new and thought-provoking in this post.
Hi, I am Chris Voegeli, and I’m an evaluation fellow in the Center for Surveillance, Epidemiology, and Laboratory Services at the Centers for Disease Control and Prevention in Atlanta, Georgia. Over the last year, my Canadian Evaluation Society colleagues Gene Krupa, Barbara Szijarto, Margo Schmitt-Boshnick, and I reviewed the literature and used our personal experience to think critically about Evaluation Advisory Groups (EAGs). Before joining the CDC Evaluation Fellowship Program, I led the National Evaluation Center for the National Network of STD Clinical Prevention Training Centers. While at the National Evaluation Center, we used several types of EAGs in our work regularly. I learned that the purpose of an EAG guides the planning and implementation of the advisory group. It’s important to identify the right people to achieve that purpose. If you are looking to enhance or support the methodological or technical quality of the evaluation, recruit seasoned evaluators familiar with the evaluation of programs similar to yours. If you are interested in generating regular and rich input in decisions throughout the evaluation, create an EAG of representatives of key stakeholder groups. When you have multiple funders with competing interests or informational needs, it can help to have an EAG that brings funders together to talk about the evaluation or address evaluation-related issues as they arise. Perhaps you need an EAG that meets another purpose or combination of purposes.
Hot Tips:
As you consider the purposes, make-up, and structure of an advisory group, consider these important questions:
- What do you hope to accomplish by adding an EAG, and is an EAG a good way to meet that aim?
- How many people or represented organizations will ensure that you receive robust and usable feedback?
- Do you have adequate human or fiscal resources to establish and maintain this EAG?
- Will differences in power among participants bias or stifle discussion or recommendations?
- How will you operate the EAG to meet your aims and ensure participants feel valued and appreciated? Will you need a charter, by-laws, a conflict-of-interest mechanism, and a written scope of work?
Rad Resources
- Developing and Using an Evaluation Consultation Group is a primer on EAGs prepared by Michael Baizerman and colleagues at the CDC. This open-access resource addresses crucial questions about how an EAG contributes to better and more useful evaluations, how to improve an EAG, and more.
- New Directions in Evaluation dedicated an entire issue to EAGs in 2012. For example, the issue includes a review of the literature relevant to EAGs, discussion of types of EAGs and other forms of evaluation advice, and case examples.
- In October of last year, Barbara Klugman wrote a compelling post for this blog on establishing an EAG when your evaluation task requires more skills than you have.
EAGs are complex and, like many other aspects of evaluation work, include potential pitfalls. Preparation is important. These resources will help you plan for the worst and aim for the best!
The American Evaluation Association is celebrating Gov’t Eval TIG Week with our colleagues in the Government Evaluation Topical Interest Group. The contributions all this week to aea365 come from our Gov’t Eval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.