Contactless Evaluation – (in Times of COVID-19) by Vardhani Ratnala

Hi, hope you are having a good weekend. This is Vardhani Ratnala, an evaluation consultant. Reading the latest news on COVID-19, specifically the relief packages announced by governments, I can’t help but wonder about work prospects for Evaluation Consultants like me.  With travel restrictions and lockdowns imposed due to coronavirus, livelihoods of many evaluation professionals have been affected. Evaluation assignments seem to be drying up, as organizations are no longer willing to or cannot evaluate their projects. The question uppermost on everyone’s mind is: how do you evaluate when you cannot travel, cannot visit target areas or target communities, and when you have to practice social distancing?

What is the solution? Should all organizations postpone evaluations for another 3-6 (or in the worst-case scenario 12-18) months until the pandemic is hopefully controlled? – No. In my opinion there are some projects and programs that can still be evaluated using Contactless Evaluation.

So, what are the ways in which we can do a Contactless Development Evaluation? 

1. Desk-based evaluation: Evaluators can do a thorough desk review of all project documents including quantitative and qualitative data. Information gathered from this review, combined with literature review and web search can help in evaluating some projects.
2. Evaluation using online data collection tools: If your target respondents have access to technology (internet or phone), it is feasible to reach out to them and collect information through phone calls, video calls, email questionnaires and online survey tools like google forms, survey monkey etc.
3. Theory-based online evaluation: In this type of evaluation, the evaluator reads through project documents and constructs the project’s theory of change (ToC). In case the project already has a ToC, it can be revised to show how the project was originally envisaged versus how it was implemented. Similarly, the project’s logframe should also be updated to show targets versus achievements. Then, virtual workshop(s) or individual discussions can be organised with key stakeholders to present and discuss the planned and final versions of the ToC and logframe. The discussion(s) will generate a lot of information for the evaluator(s) to evaluate the project.
4. Contactless evaluation through Peer Evaluators: For real time evaluations like emergency response, if there is an urgent need for data collection but phone/internet connectivity is limited; evaluators can collect data through peer evaluators. For example: To evaluate services provided at a relief camp, few of the camp residents with access to phone/internet can be trained online on simple evaluation tools and asked to collect data from other residents maintaining social distance and following principles of hygiene.

Hot Tip: Sometimes a mix of all four types of Contactless Evaluation might be needed. Also, it is important to note that Contactless Evaluation doesn’t work for all projects, specially those where your respondents do not have access to technology, where you need to use participatory data collection techniques, or where you need a rigorous evaluation methodology.

Now it’s your turn: Are there any other styles of Contactless Evaluation you’ve come to appreciate during this time? Share with us in the comments below or in our Evaluators’ Slack Channel, where you can comment, share links, and even upload resources. It’s easy to join and free to use. We’ll see you there!

---

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.