Enhancing Learning by Requiring Written Project Responses by Stephanie Chasteen

Hello, AEA365 community! Liz DiLuzio here, Lead Curator of the blog. This week is Individuals Week, which means we take a break from our themed weeks and spotlight the Hot Tips, Cool Tricks, Rad Resources and Lessons Learned from any evaluator interested in sharing. Would you like to contribute to future individuals weeks? Email me at AEA365@eval.org with an idea or a draft and we will make it happen.

---

I’m Stephanie Chasteen, an independent consultant who serves as external evaluator on NSF-funded projects.

One of the biggest challenges as an evaluator is to make sure that your results go to good use. As Michael Quinn Patton once said, “The purpose of an evaluation is not to produce a report.” Rather, it is to produce learning that leads to improved outcomes. Yet, a lot of that learning is encompassed in the report. So how can you help your clients use these reports for good?

The most important thing that I have started to do is to require a written response to my evaluation reports. This was not common practice with my clients and has been a game changer. I got this idea from an AEA post by Kylie Hutchinson (thanks Kylie!)

Lessons Learned

Have a deadline and a common space for submitting the response

I used to include an action planning template in the final page of my report. Nobody ever used it. So now here’s how I do it:

1. I send them the final written report and a deadline for a response. I typically produce 4-5 evaluation reports/year for context.
2. I have a “Project responses to evaluation feedback” Google Doc where I list the report, the key recommendations, and put an empty box for their responses with the deadline. (Note that this document also serves as an index to all evaluation reports).
3. They write up their response within a few weeks. My best clients do this as part of a team meeting, and thus the evaluation report has served as an artifact for them to engage around and make meaningful plans for their project.
4. We meet to discuss.

Don’t micromanage it

I don’t micromanage the format of their responses; some write bullets, some write narratives, but it’s usually 2-5 paragraphs. I used to give a table with the person responsible and deadline, but that felt too daunting for people. If I feel that they’ve left something out, I poke at them for it. Otherwise, I just say ‘great job.’ I do make sure those responses have generated action items on their individual to-do lists. If I really feel they’ve missed the mark, I will sometimes show them examples from other clients.

Use this for the year-end report

At the end of the year, I review those project responses. I highlight to the NSF how much my client has engaged in continuous improvement and what kinds of changes they made in the project in response to the evaluation. And I highlight to the client some things that I think they may need to circle back to.

Use Google Doc reports

I often give my reports as Google Docs and invite commentary using the comments feature. This makes sensemaking on the report more collaborative and real-time and supports learning.

Consider a 6-month evaluation audit

For some clients I will do a 6-month “evaluation audit” where I look through the evaluation reports and project responses and write a memo of things to take another look at. I got this idea from another AEA post.

My experience has been positive

1. Clients don’t grumble about doing this. I present it as normative, and they take it as such.
2. Clients quickly realize how valuable this is.
3. This process encourages both single-loop and double-loop learning.
4. This process creates accountability for responding to evaluation findings, and for acting on those plans.

Consider using an evaluation report protocol

For one client who has a very complex organizational structure and there have been concerns about report dissemination, I have an agreed-upon plan for evaluation report roll-out and responses as shown below.

Rad Resources:

1. Evaluation Communication Plan Checklist from Evalu-ATE.org
2. Post-evaluation action planning AEA 365 post from Kylie Hutchinson
3. Following up to drive change AEA 365 post from Janet Mou Pataky and Diana Tindall

---

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.