Hi! I’m Natalie Donahue. I am the Chief of Evaluation in the State Department’s Bureau of Educational and Cultural Affairs’ Monitoring Evaluation Learning and Innovation (MELI) Unit and am the Washington Evaluators (WE) Past President.
The Evidence Act has had a great impact on federal evaluation practices. Over the past five years we’ve seen federal agencies create learning agendas, increase capacity-building efforts, update (or, in some cases, create) evaluation policies and accompanying guidance documents, and increase collaborative efforts around evaluation – both internally and with other agencies.
Within Washington Evaluators, there have been concerted efforts to incorporate the Evidence Act into the organization’s programs; for example with the Five Things to Know about Evaluation Policy in 2018 and Creating an Evaluation Capacity Building Leadership Framework. These programs have provided our members with valuable information about the Act itself, how other WE members are responding to the Evidence Act’s mandates, and resources to enhance monitoring and evaluation efforts in an organization/agency.
At the working level, the Evidence Act has served as the foundation for evaluators to be able to implement new initiatives that might not have been so readily approved prior to the Act, and has reinforced the need for decision-makers to more thoughtfully (and routinely) integrate data into their decision-making.
The Evidence Act provided the impetus for my colleagues in the MELI Unit and I to think more critically about the learning aspect of our efforts. In 2019, we created the bureau’s first learning agenda and have continued to develop strategic learning resources – learning briefs to summarize evidence around a learning agenda question, webinars on what a learning agenda is, and our Evaluations Matters publications that demonstrate how evaluation data have been used – to ensure everyone in the bureau has access to the information and data they need, and are using the data to inform any programmatic decisions they may need to make.
Lessons Learned
- Every four years my team works with our bureau to create a new strategic plan. Instead of creating an entirely new process for a learning agenda, we just added another session to the strategic planning process to discuss the learning agenda.
- We made our learning agenda creation process simple for our non-evaluator colleagues by using simple terms (i.e. by asking them about gaps in our bureau’s collective knowledge; versus telling them “Hi. We are here to create a learning agenda and this is what that is”). By using a pre-existing process and limiting the jargon, we were able to get the feedback we needed to create our learning agenda with no issue.
Hot Tips
Our first learning agenda sat on the shelf, so to speak. With the creation of the Learning Agenda to match the new FY22-26 strategy, we updated our processes and created a number of new products to ensure that we were incorporating the learning agenda questions and lessons learned throughout all of our work. These include:
- Updated standard operating procedures requiring evaluations to include data collection questions related to relevant learning agenda questions;
- A repository for evaluation managers to include all relevant information from the evaluation that relates to the various learning agenda questions
- The creation of Learning Briefs, which allow us to share learning around specific topics with audiences (example here); and,
- Strategic learning resources like our webinar on what a learning agenda is and how we plan to incorporate it into our bureau’s learning.
Rad Resources
- All of MELI’s learning agenda-related resources are public and can be found here.
- You can dig into the materials of past WE events on the website.
The American Evaluation Association is hosting Washington Evaluators (WE) Affiliate Week. The contributions all this week to AEA365 come from WE Affiliate members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.