I am Lycia Lima, the executive coordinator of the newest CLEAR center- for Brazil and Lusophone (Portuguese-speaking) Africa. We’re formally joining CLEAR later this year and are planning our inauguration in October 2015. I was also one of the organizers involved in the formation of the Brazilian M&E Network – Rede Brasileira de Monitoramento e Avaliaçã – which has become a very active association.
We’re based in Brazil, at the Sao Paulo School of Economics at Fundação Getulio Vargas and work jointly with the school´s Center for Applied Microeconomics. Through CLEAR we’re looking forward to expanding into new areas and building bridges with the M&E communities in Brazil and elsewhere. In particular, we’ll be working to advance evaluation capacity development services and products in Portuguese for use in Lusophone (Portuguese-speaking) countries, all to foster evidence-based policy making in these countries.
Historically, our team in Brazil has had a lot of experience in carrying out impact evaluations in all sectors. Though we specialize in impact evaluation, we have experience in and appreciate the broader range of M&E approaches, and think that an integrated approach will make our work better. In this post, I have put together a few tips about impact evaluation that you would not learn in conventional econometrics books. This is advice I’d give to impact evaluators.
Lessons Learned: Know well the theory of change of your intervention! If you don´t know the theory of change well, you might not fully understand the causality channels and might leave important impact indicators out of the analysis. Get your hands dirty! Go to the field, talk to project managers, talk to beneficiaries and make sure you fully understand the intervention you are trying to evaluate. Also, be careful with the quality of your data. Make sure you spend some resources on hiring and training qualified staff to supervise data collection. Good quality data is crucial for your study.
Lessons Learned: Even if you are an empiricist and believe mostly in quantitative methods, do not underestimate the value of mixed methods. In particular, qualitative approaches will help you understand “why and how” things happened. Importantly, get to know M&E “foundational” literature from Patton, Scriven, Bamberger, and others.
Rad Resources: While in general M&E materials available in Portuguese are limited in numbers, there is a very useful impact evaluation book that I have co-authored with other Brazilian experts. The book may be obtained free at
http://www.fundacaoitausocial.org.br/_arquivosestaticos/FIS/pdf/livro_aval_econ.pdf
We look forward to contributing to the M&E literature base in Portuguese, so please check back with us on this.
The American Evaluation Association is celebrating Centers for Learning on Evaluation and Results (CLEAR) week. The contributions all this week to aea365 come from members of CLEAR. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Good point about the value of qualitative data. I would add, in addition to telling us *how* and *why* something happened, qualitative data can also tell us *what* happened. In my experience, qualitative data like individual and focus group interviews, structured observation, and analysis of open-ended survey questions often reveals program impacts and benefits that leaders did not know about before.