Why (Real) Participation Is A Still a Pipe-dream (So Far ) In My Evaluation Practice by Sara Vaca

Hi, AEA365 family! I’m Sara Vaca (frequent Saturday contributor to this blog), independent consultant specializing in evaluation of international development programmes and policies, and I’m platonically in love with Participation. Here is my practical relationship with it so far, though.

Hot Tip: I understand participation as including many different actors, preferably those who have less voice, not as informants, but as part of the group that make decisions about the design and process, rather than just commissioners and evaluators. As a constructivist trying to bring an Equity and Gender focus to my evaluations, I understand participation enriches (or even legitimates) the process and results. So, since my first evaluations, I always pitch the idea of including beneficiaries (and other actors) into the evaluation team.

And every time I am about to face a new, during the inception phase my hopes go high on trying to make it really participatory. But after many unsuccessful attempts, my experience has kind of killed my hopes and I had that wise, yet sad realization: “It is most probably not going to work, as usual…”.

Lessons Learned:  These are the main obstacles I have found in my practice that make the commissioner organization and the evaluation managers (and sometimes even myself) decline my suggestions to foster participation:

• Organizations do not understand participation as bottom-up: even when the Terms of Reference talk about high levels of participation, they often have been elaborated “in-house” (= without participation)
• Normally, participation makes the evaluation more expensive (unaffordable?): it entails logistics costs to make actors meet in centered places, to participate in meetings where things related to the evaluation are decided and defined, and longer times (more expensive again, in terms of evaluator’s fees and costs).
• Beneficiaries of international development programmes usually do not know what evaluation is: in contexts of low-income, low-literate, urban or rural communities who have problems to survive every day, involving them in a technical discussion about what to ask and how to answer evaluation questions is forced, hard and almost irrelevant to them.
• Ultimately, given the aid-vicious-circle and hardship they face, beneficiaries are busy and often engage into the process solely with the hope of getting extra aid, that is, the more they engage, give their time, the more they expect that will revert into new projects supporting them or their area.

Rad Resource: Forming Evaluation Reference Groups where you invite authorities, communities representatives and other organizations to comment on the ToRs, evaluation design and sampling, or at least to participate in a Findings Validation Workshop, are so far the two realistic mechanisms that in my view grant some participation minimums.

Cool Trick: However, I often use a Participation Scanner to depict the participation levels of the evaluation. And yes, until the majority of squares (or at least the ones at the right) is dark, no matter how many groups the evaluation consulted as informants, it is still not participatory.

Does the same happen in other contexts or types of evaluation? 🙂

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on theaea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.