Hello, my name is Pedro Mateu and I am a second year student in the Interdisciplinary Ph.D. in Evaluation program at Western Michigan University. During last spring I learnt how to do meta-analyses as a part of the program’s core courses. Therefore I would like to share with you some lessons learned.
Hot Tip: First of all, let me define meta-analysis and why it is important. According to Gene V. Glass, meta-analysis seeks to integrate conclusions across multiple studies and disciplines by applying statistical analyses to groups of studies. Through meta-analysis, evaluators can estimate not only the central tendency of study outcomes, test the pattern of outcome variations, and estimate the overall effects and relationships of variables, but also predict results of future evaluations. The application of this tool is valuable because meta-analysis yields objective, defensible, and largely value-neutral evidence, which policy- and decision-makers could reference when forming and revising policies and programs.
Lesson Learned: Conducting a meta-analysis consist of two parts: a coding process and a statistical analysis. Even though the second part sounds to be more complicated, do not underestimate the first one. During the first part, the coders have to spend time to develop the best code form that will be helpful to gather all key data from each study. As a consequence, a first lesson learned is the great value of creating and developing a database which contains the coded studies.
Lesson Learned: A second lesson learned after conducting a meta-analysis is about the type of studies that can be included into the analysis. They must be primary ones. They cannot be research letters, editorial letters, commentaries or pre-publication articles. In order to prevent this, the coders who analyze the data of studies must be accurate when they read the articles.
Both lessons learned are the key ones that are needed to conduct a successful meta-analysis.
Rad Resource: In regard to the statistical part, I suggest you to check the online Cochrane Handbook for conducting meta-analyses available in the next link: http://www.cochrane.org/training/cochrane-handbook. An important fact in the title of this book is the name that the Cochrane Collaboration uses for meta-analyses. It is replaced by systematic reviews.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Want to hear more from Pedro? Attend the Meta-analysis Demystified workshop he’ll be presenting with colleagues during AEA’s Annual Conference this November – Evaluation 2011. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.