Enhancing Learning by Requiring Written Project Responses by Stephanie Chasteen

Hello, AEA365 community! Liz DiLuzio here, Lead Curator of the blog. This week is Individuals Week, which means we take a break from our themed weeks and spotlight the Hot Tips, Cool Tricks, Rad Resources and Lessons Learned from any evaluator interested in sharing. Would you like to contribute to future individuals weeks? Email me at AEA365@eval.org with an idea or a draft and we will make it happen.


Stephanie Chasteen

I’m Stephanie Chasteen, an independent consultant who serves as external evaluator on NSF-funded projects.

Double Loop by icon 54

One of the biggest challenges as an evaluator is to make sure that your results go to good use. As Michael Quinn Patton once said, “The purpose of an evaluation is not to produce a report.” Rather, it is to produce learning that leads to improved outcomes. Yet, a lot of that learning is encompassed in the report. So how can you help your clients use these reports for good?

The most important thing that I have started to do is to require a written response to my evaluation reports. This was not common practice with my clients and has been a game changer. I got this idea from an AEA post by Kylie Hutchinson (thanks Kylie!)

Lessons Learned

Have a deadline and a common space for submitting the response

I used to include an action planning template in the final page of my report. Nobody ever used it. So now here’s how I do it:

  1. I send them the final written report and a deadline for a response. I typically produce 4-5 evaluation reports/year for context.
  2. I have a “Project responses to evaluation feedback” Google Doc where I list the report, the key recommendations, and put an empty box for their responses with the deadline. (Note that this document also serves as an index to all evaluation reports).
  3. They write up their response within a few weeks. My best clients do this as part of a team meeting, and thus the evaluation report has served as an artifact for them to engage around and make meaningful plans for their project.
  4. We meet to discuss.
Don’t micromanage it

I don’t micromanage the format of their responses; some write bullets, some write narratives, but it’s usually 2-5 paragraphs. I used to give a table with the person responsible and deadline, but that felt too daunting for people. If I feel that they’ve left something out, I poke at them for it. Otherwise, I just say ‘great job.’ I do make sure those responses have generated action items on their individual to-do lists. If I really feel they’ve missed the mark, I will sometimes show them examples from other clients.

Use this for the year-end report

At the end of the year, I review those project responses. I highlight to the NSF how much my client has engaged in continuous improvement and what kinds of changes they made in the project in response to the evaluation. And I highlight to the client some things that I think they may need to circle back to.

Use Google Doc reports

I often give my reports as Google Docs and invite commentary using the comments feature. This makes sensemaking on the report more collaborative and real-time and supports learning.

Consider a 6-month evaluation audit

For some clients I will do a 6-month “evaluation audit” where I look through the evaluation reports and project responses and write a memo of things to take another look at. I got this idea from another AEA post.

My experience has been positive
  1. Clients don’t grumble about doing this. I present it as normative, and they take it as such.
  2. Clients quickly realize how valuable this is.
  3. This process encourages both single-loop and double-loop learning.
  4. This process creates accountability for responding to evaluation findings, and for acting on those plans.
Consider using an evaluation report protocol

For one client who has a very complex organizational structure and there have been concerns about report dissemination, I have an agreed-upon plan for evaluation report roll-out and responses as shown below.

Rad Resources:

  1. Evaluation Communication Plan Checklist from Evalu-ATE.org
  2. Post-evaluation action planning AEA 365 post from Kylie Hutchinson
  3. Following up to drive change AEA 365 post from Janet Mou Pataky and Diana Tindall

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.

1 thought on “Enhancing Learning by Requiring Written Project Responses by Stephanie Chasteen”

  1. Hi Stephanie,

    Thank you for your post about how to enhance learning from evaluation projects by requiring written responses. You’ve emphasized that the purpose of the report is not just to disseminate information about the project, but to take this information, run with it, and improve the program to hit a broader, more effective impact on the target audience. This post reminds me of another blog post I read on AEA365 by Pam Lilleson: “How to Make Evaluation Results Actionable for Decision-Makers”. It is about making evaluation results actionable by decision makers. She mentions similar notions, that in order for evaluation to be effective, you need to make sure stakeholders and decision makers are involved. While she mentioned as an evaluator to involve yourself in the decision making meetings, I like that you’ve made it imperative that decision-makers are involved in your process. Having written project responses requires a client to go back to the team, get a consensus on the feedback on the evaluation report, and then put it into words for you. The other blog post also mentioned that a report needs to be accessible to the client, meaning, it can be in any form that makes the report more accessible. You’ve taken it a step further to include double-loop learning, and actually engage in the accessible report. I like it!

    Thank you for sharing!
    Z.

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.