AEA365 | A Tip-a-Day by and for Evaluators

TAG | evaluator role

Hi! We are Courtney Bolinson and Muthoni Wachira, Impact Evaluation Manager and Investment Director at Engineers Without Borders Canada (EWB). EWB invests in seed-stage social enterprises to help them scale and drive lasting, transformational impact within communities that need it most. EWB recognised the need for an evaluator to measure and interpret the impact of investments, however integrating an evaluator into a team unfamiliar with evaluation was challenging, and we learned some useful lessons about integrating our two areas of work.

Lesson Learned: Get on the same page. During one of the first retreats with the investment team, Courtney gave an introductory presentation on program evaluation, provided definitions, shared standards and guiding principles, and summarized the state of evaluation in the impact investing sector. This was a turning point for our team’s understanding of the evaluator’s role.

Lesson Learned: Work together to identify where evaluation can be used. At first, Courtney was unfamiliar with impact investing, and focused on summative evaluation. Over time and through collaboration, we were able to identify a number of formative and implementation evaluation needs at different stages of an impact investment, such as due diligence.

Hot Tips:

  1. If you’re new to impact investing, have an expert from your team or organization walk you through the steps of an investment process. This will help you identify areas where evaluation can be useful.
  2. Use the full spectrum of an evaluator’s skillset. An evaluator is well-placed to help clarify an investment theory of change, develop an impact thesis, and identify key performance indicators for the investment fund.
  3. Position your evaluator as a key member of the team. For example, ensure they are on weekly calls, strategy meetings, team retreats, etc. This will help them understand the context of their evaluation work better and will allow for the team to identify new areas for evaluation.
  4. Read each other’s literature. For an evaluator, reading impact investing blogs, state of the sector reports, and investor-specific impact reports is critical for providing evaluation context. For an investment director, reading evaluation literature can clarify what is possible regarding evaluation, and what is considered cutting edge.

Rad Resources:

The American Evaluation Association is celebrating Social Impact Measurement Week with our colleagues in the Social Impact Measurement Topical Interest Group. The contributions all this week to aea365 come from our SIM TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

I am José M. Díaz-Puente, professor of the Technical University of Madrid (Universidad Politécnica de Madrid–UPM) where I am member of the Research Group GESPLAN on Sustainable Planning and Management of Rural/Local Development. I have worked with participatory and empowerment evaluation approaches in rural areas of Europe and Latin America.

I would like to share with you a few ideas based on my experience regarding the goal and evaluator’s role when applying an empowerment approach in the evaluation of rural/local development programs.

Goal: The empowerment evaluator has to be focused on facilitating: A leading role and full responsibility of the stakeholders with regard to the internal evaluation system and all monitoring tools; and a collaborative role and some responsibility with regard to the summative evaluations.

Hot Tip:

Evaluator’s role: Evaluator’s roles have been described with all these words: facilitator, educator, consultant, counselor, collaborator, critical friend, interpreter and mediator, among others.  Each of these terms adds something to the evaluator’s involvement in the evaluation process. In my experience of applying EE with development programs, I usually define the role of evaluation professionals with two words:

  • Evaluator, which refers to professionalism and, in turn, provides credibility to the evaluations made. It is the role that leads to meeting the donor’s requirements (production of credible evaluation documents) and is more related to the evaluation findings.
  • Facilitator, which refers to the establishment of evaluation capacity among the participants (i.e., their role as educators, collaborators, critical friends and counselors). It is the role that leads to the building of capacity in the stakeholders to undertake their own internal evaluation process and is more related to the process of evaluation.

Rad Resource:

  • Article on EE: Evaluation Review

Díaz-Puente, J.M., Yagüe, J.L., Afonso, A. (2008) Building Evaluation Capacity in Spain: A Case Study of Rural Development and Empowerment in the European Union. Evaluation Review, 32(5), 478-506.

It is a case study from Spain that provides an example of the diffusion of evaluation culture through the use of an empowerment evaluation approach to build evaluation capacity within the context of rural development.

I hope these ideas can be of any use to you to reflect on your own work with EE.

The American Evaluation Association is celebrating Collaborative, Participatory & Empowerment Evaluation (CPE) Week with our colleagues in the CPE AEA Topical Interest Group. The contributions all this week to aea365 come from our CPE members and you may wish to consider subscribing to our weekly headlines and resources list where we’ll be highlighting CPE resources. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice.

·

Archives

To top