My name is Robert McCowen and I am a doctoral fellow in Western Michigan University’s Interdisciplinary Ph.D. in Evaluation. I served as a session scribe at Evaluation 2010, and attended session number 606, Fidelity of Program Implementation in Educational Evaluations. I attended the session because my evaluation interests focus on education and assessment, and a recent research project with colleagues has involved overcoming—not always successfully—several obstacles to treatment fidelity.
Lessons Learned: Hendrick Ruitman of Cobblestone Applied Research and Evaluation, Inc., offered tips for evaluators who are attempting to ensure fidelity of implementation:
- Manage the expectations of those responsible for implementation. If stakeholders expect a magic bullet, they’ll surely be disheartened.
- Standardize training for implementers, and when possible make sure the trainers can discuss evaluation practices during training. This helps keep everyone on the same page—and makes evaluation a routine part of the program or treatment, rather than a later imposition.
- Make an effort to keep implementation guidelines handy for the people who have to use them, and make the guidelines noticeable. A laminated sheet of hot pink paper is easier to find and reference than a pamphlet or notebook.
- Develop an accurate way to track fidelity before the evaluation begins. Ad hoc methods work exactly as well as any experienced evaluator would expect!
- Including fidelity of implementation in your evaluation design can help highlight failures and areas for improvement, and gives a great deal of information and control to stakeholders.
Hot Tip: Two authors have written about measuring fidelity on aea365. See their posts for more on this topic:
- Karen Elfner Childs on Measuring Fidelity of School-Wide Behavior Support
- Elizabeth Oyer on Promoting Implementation Fidelity: Tools to Support Local Capacity
At AEA’s 2010 Annual Conference, session scribes took notes at over 30 sessions and we’ll be sharing their work throughout the winter on aea365. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice.