Hey there! I’m Martena Reed, Research Associate at the Urban Institute where I study community engagement and solutions for homelessness. I’m interested in and always exploring ways to thread equity into the methods we use to understand implementation and outcomes of social change efforts.
As shelter-in-place orders are being implemented across the country, we as evaluators are faced with a question: how should we move forward with existing evaluations? Many of us find ourselves thinking about ways to proceed that are less burdening to practitioners who are in the trenches of the COVID-19 emergency response and to community members who are also navigating a new normal.
Rather than a knee-jerk reflex, COVID-19 is an opportunity for evaluators to reflect more deeply about the methods we use. Here are few lessons I’m taking from the times we’re in:
- Develop evaluation contingency plans. We’re often providing funders an evaluation plan that includes best case scenario and prioritize in-person interactions. Rarely do plans require the evaluation team to offer contingency options. As many of us are now tailoring methods to respond to social distancing and travel recommendations, we’re switching to virtual interviews and other methods of web-based data collection. Building contingencies into future evaluation plans will leave us better prepared to pivot and could save time and resources on creating post-hoc plans.
- Include low-interference, creative evaluation methods. Interviews, focus groups, and surveys are common primary sources for evaluation data, often happening in-person, but the pandemic places evaluators in a position to explore and deploy other methods. Journaling and photo voice methods allow community members and practitioners to capture important evaluation data and how they’re experiencing COVID-19 (as many are adding evaluation questions that capture this!). There’s always more traditional methods like document review – consider asking practitioners to share their team Box folder with you at the beginning of the evaluation, and you could see, in real-time, all the documents they’re using right now or changes they’re making to existing policies.
- Build in time for human connection. While the COVID-19 pandemic is something that we’re all experiencing together, there are plenty of times when the people with whom we’re collecting data are in the middle of a distressing life event. We’ll all be better if we keep building in time for human connection. Whether it’s at the kick-off of a project or the beginning of an interview, leave space for building relationship, checking in with one another, and connecting. Ultimately, the relationships help to build trust that leads to a stronger and more reliable evaluation.
Although the current crisis may lead to some new thinking, these are really issues we need to be reflecting on and considering in our evaluation all the time. If you’re in the process of pivoting evaluation plans in response to COVID-19, consider these resources from NVivo on virtual methods and this thoughtful World Bank blog post from Development Impact.
Now it’s your turn: What are some lessons you’ve learned during this time? Share with us in the comments below or in our Evaluators’ Slack Channel, where you can comment, share links, and even upload resources. It’s easy to join and free to use. We’ll see you there!
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.