Hello, AEA365 community! Liz DiLuzio here, Lead Curator of the blog. This week is Individuals Week, which means we take a break from our themed weeks and spotlight the Hot Tips, Cool Tricks, Rad Resources and Lessons Learned from any evaluator interested in sharing. Would you like to contribute to future individuals weeks? Email me at AEA365@eval.org with an idea or a draft and we will make it happen.
I am Maggie Jones, Director of the Center for Community Health and Evaluation. I recently did a panel presentation at AEA’s 2025 conference with Maryam Khojasteh and Jess Renger, colleagues from Robert Wood Johnson Foundation and Kaiser Permanente’s National Office of Community & Social Health, on how our role as a “stand-by” evaluator has enhanced organizational learning and adaptability within their organizations. Given the robustness of the panel discussion, we want to share our lessons learned.
What do we mean by stand-by evaluator?
We have an open and flexible evaluation contract to support emergent priorities related to evaluation, measurement, planning/design, and organizational learning. The work may entail a variety of projects including cross-program synthesis, learning strategy, program planning, evaluation planning, and evaluation projects.
How does it differ from traditional evaluations?
Lessons Learned
- Ensure strong communication, coordination, and alignment between the funding partner and evaluation team.
- Establish roles and identify primary points of contact for both the evaluation team and funding partner.
- Identify a core evaluation team that will be accountable for all projects in the emergent portfolio.
- Build in time for relationship development and check in regularly for feedback, course corrections, approvals, real?time alignment, and (re)prioritization as needed.
- Prioritize open communication and discuss challenges as they emerge. Structured reflection helps identify challenges and tension points in the work and lessons across projects.
- Build structure for transparency around priorities and progress
- Establish a mechanism for identifying, vetting, and prioritizing potential projects – a menu of options can be a helpful starting point.
- Maintain a dashboard of all ongoing and recently completed projects to ensure shared understanding of priorities and progress.
- Create project folders with shared online access.
- Develop consistent documentation of core materials to ensure all projects have a parallel structure.
- Track and be transparent about use of financial resources to inform priorities and the pace/intensity of the work.
- Focus on scaffolding – don’t overbuild infrastructure because things will change.
- Anticipate need for evaluation team flexibility in managing emergent requests
- Be willing to pivot, open to emerging scope of work, and have a breadth of expertise to be able to plug into different types of requests without significant ramp up. The team should be able to bring in additional staff and expertise for specific projects when needed.
- Plan for the pace of various projects to align with funder priorities/ timelines and evaluation team capacity. Have some larger projects/workstreams to ensure sufficient capacity, and maintain a list of “nice to have” or non-urgent priorities to pick up during slower times
- Allocate time for co-development of work/priorities
- Work often starts with an idea or a question, which means the evaluation partner is coming in much earlier. It takes time to translate ideas into actionable workstreams, so plan accordingly! Work is also sometimes experimental and won’t always be linear.
- The evaluation team will require funder support to navigate and prioritize work, so the funder must be willing to share internal processes and tensions. The evaluation team can provide “external” insights more effectively if they fully understand the situation and context.
- The longer the partnership, the more independent the evaluation team will be because they will have a deeper understanding of the organization.
Our funding partners have shared that engaging with a “standby” evaluator has contributed to more robust cross-program and initiative learning to inform strategy, expanded capacity for smaller evaluation projects, brought external perspective to internal decisions, and allowed for exploration and experimentation to test new ideas and approaches.
Our slides from the recent AEA panel, including responses to participant questions can be found on our website.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.