Hello! I am Terence Fitzgerald, Senior Director of Program Design & Evaluation at International Justice Mission (IJM), a nonprofit organization that works to protect people from violent crime. I lead a team of staff that provides leadership and technical support to IJM on program design, monitoring, evaluation, and research. My team’s remit is to “bring evidence to bear for the mission” so that leaders at executive, portfolio, and project levels can make informed decisions; and so that IJM can realize its vision and fulfil its mission. I want to share lessons that my team has learned from working on program design from inside a mission-driven organization.
- Demystify program design: Before we engage with a team on program design, we ask for and listen to their views and experiences of it; and, where necessary, we explain concepts, processes, definitions, benefits, and challenges. We meet the teams where they are. I have found that a one-hour session on designing a personal program around “being more healthy” can generate lots of input and allow for discussion of many core design issues – different types of impact; relationships between resources, activities, and results; milestones and indicators; and assumptions and risks. Even with inexperienced staff, it can be quite easy to convey key concepts, get enthusiastic buy-in, and then build a more solid design.
- Progressively elaborate the design: As we work with teams on a design, my team creates draft logic models and other design artifacts based on our understanding of what they have told us. We give those back to the team to review our work, correct any erroneous content, note any unresolved issues, and to secure buy-in for further elaboration. We raise concerns where we see threats to the design’s plausibility or feasibility, and we help teams to understand the causes of our concerns. We raise unresolved concerns to decision-makers.
- Facilitate change to designs: Teams can tend to revere and thus be reluctant to change the designs into which they have put time and energy. My team reinforces that designs are meant to change, based on the team’s experiences and environmental changes. We explain and, where necessary, guide teams through IJM’s program change management process.
- The OECD DAC criteria for evaluating development assistance are very useful to bring evaluation into the program design process. My team uses the criteria to inform design discussions that often lead to clarifications on program priorities, inclusions and exclusions, and other design elements.
- Rachel Kleinfeld’s Improving Development Aid Design and Evaluation: Plan for Sailboats, not Trains is an excellent resource for those designing complex programs centered on political change. It highlights several design errors that my team has seen and that we seek to avoid.
The American Evaluation Association is celebrating Program Design TIG Week with our colleagues in the Program Design Topical Interest Group. The contributions all this week to aea365 come from our Program Design TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to firstname.lastname@example.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.