Deeply Involving Clients in Designing and Executing Evaluation Studies by Ryan Evans

My name is Ryan Evans and I’m a research associate at Wilder Research, a nonprofit research firm based in Minnesota. At Wilder Research, I work primarily with small- and medium-sized nonprofits in the Twin Cities. When working with smaller clients, it is paramount to deeply involve them in planning and doing the evaluation to ensure that the results are as useful as possible for them.

Lesson Learned: When I started my career as an evaluation consultant, I designed cookie-cutter evaluations. A survey, some focus groups – or both in a mixed methods design – that culminated in a report. I’ve learned that cookie-cutter evaluations are often not responsive enough to the context and changing circumstance of small nonprofits to provide useful results. I have evolved my consulting style to deeply involve my clients in my evaluation work, resulting in an increased likelihood that they can use the results to strategically guide their organization.

Hot Tip: Use an iterative approach. When working on evaluation projects, I will modify my project plan to respond to new ideas that arise from planning and doing the evaluation. I repeatedly ask myself and my client, “Is this work meeting our learning goals? Will this work be useful for improving the program and increasing its reach and sustainability? What might be more useful?” For one of my projects, I had completed half of the planned interviews. When talking with my client about the findings so far and how they related to the project’s learning goals, we decided I should also observe their programming – so we canceled the remaining interviews and I observed the program instead.

Cool Trick: To expedite the iteration process, give clients something concrete and fairly detailed to respond to – a draft infographic, for example – as early as possible. I spend a relatively small amount of time developing initial drafts so that I receive feedback from my clients quickly. This speeds up the process immensely (compared to waiting until I feel I have developed something “just right”).

Hot Tip: Build on the expertise of your clients. I am working with a theater organization and recently proposed doing a student perception survey. They didn’t like the idea of doing a written survey because it wouldn’t utilize their expertise or preferred approach as theater artists. Instead, we designed a “talking survey” that they facilitated with their students. I designed the survey and took notes as they talked through the questions with their students and interactively obtained the data we wanted.

Rad Resources: In my informal researching, the consulting field calls this consultation style “process consulting” or “emergent consulting.” Here’s a link to a research-based blog post about consulting styles, including process and emergent styles.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.