Hello, AEA365 community! Liz DiLuzio here, Lead Curator of the blog. This week is Individuals Week, which means we take a break from our themed weeks and spotlight the Hot Tips, Cool Tricks, Rad Resources and Lessons Learned from any evaluator interested in sharing. Would you like to contribute to future individuals weeks? Email me at AEA365@eval.org with an idea or a draft and we will make it happen.
Hi, we are Maya Lefkowich and Michaela Raab, and we are evaluators with 30 combined years of evaluation experience. Maya is an arts-based methodologist and evaluator based in Vancouver (Canada) passionate about equity-driven and joy-centred evaluation. Michaela, working from Berlin (Germany), is a senior evaluator and facilitator in international cooperation who enjoys making sense of complex projects around the world. We met online and immediately connected on the question: how can evaluators make the most of interviews?
Quality evaluations often center on strong interviews. Even predominantly quantitative approaches benefit from interviews, which add depth and nuance to findings. But, interviews must be well-facilitated to unfold their potential. And, most evaluators don’t have the time or professional development support to get there. This blog is a snapshot of our chat featuring common mistakes and lessons learned for effective interviews.
Maya: I learned to interview from workshop-style courses and jobs where my interviews were observed. Peers and supervisors provided feedback on my body language, tone, and questions. But, evaluators often work in isolation. There aren’t many opportunities for someone to observe, pinpoint problems, and offer guidance.
Lesson Learned:
To avoid the mistake of working in an echo-chamber, I developed a technique to reflect on my interviewing alone or with a colleague.
Michaela: Definitely, interviewing is a neglected discipline. For example, interview guides are often way too long. Some attempt to squeeze 50 open-ended questions into an hour-long interview, which can feel extractive and exhausting for the interviewee. Also, interview questions are often abstract and full of jargon. Some evaluation teams seem to take them straight from the evaluation terms of reference (such as, “to what extent has the intervention achieved the intended impact”), making it hard for many respondents to engage. Interlocutors who are familiar with that type of terminology may be tempted to respond in empty jargon – without reflecting on their own experience or perceptions.
Maya: Among the many interview mistakes I’ve seen evaluators make is when the interviewer treats the interview as a talking survey, chat with friends, gotcha investigation, first date, or therapy session.
Hot Tip:
Buffer enough time in the evaluation to: (a) pilot and refine the interview tool within the community to ensure that questions are wanted, meaningful, and effective; (b) build trust and reciprocity in (to avoid data extraction); and (c) practice my questions and pauses ahead of time to ensure that I let questions and answers breathe.
Michaela: Room to breathe is key! Qualitative interviews work when interviewees can relate to the questions and express themselves in their own terms. As an interviewer, I need to stay in the present moment with my interlocutor, listen and be prepared to be surprised. The point of the interview is trying to discover things I have not thought of before and elevate perspectives beyond my own.
Hot Tip:
When time is tight, (a) frame the interview as a conversation where the interviewer’s main role is to facilitate and to listen, (b) develop a short list of guiding questions (fewer than ten for a one-hour interview), and (c) use plain language.
Maya: Getting creative with questions is important too. If the meeting could have been an email, what’s the point? My advice is to position the interviewee as the expert (so they do not feel like this is a test), use creative questions to tap into hindsight (e.g., knowing what they know now) or imagination (e.g., what could be), and reciprocate their gift of wisdom with an experience that is generative and wanted. Arts-based techniques can help as well!
We have both learned (the hard way) that the interview can make or break the evaluation.
Hot Tip:
Invest in and give your interviews a chance to breathe! By investing time to hone your skills, pilot your questions, get feedback on your technique, and prioritize your craft, you can make your interviews shine. They might save your evaluation.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.
This is so useful. Thanks so much