Judy Savageau and Laura Sefton on Creating Standard Operating Procedures for Evaluation Projects

Hello! We’re Judy Savageau and Laura Sefton from the Center for Health Policy and Research (CHPR) at the University of Massachusetts Medical School. We evaluators have in our minds all the steps required to complete each project, yet we must also consider quality improvement in our work as we think about our methodologies internally and our reports/stakeholders externally. LEAN thinking, a quality solution, can be translated quite easily to research and evaluation arenas. LEAN doesn’t necessarily mean that we reduce our work to the bare minimum in order to save on limited resources – it’s a way of thinking guided by REDUCING WASTE. To do this, LEAN promotes using Standard Operating Procedures (SOPs). SOPs can result when we consider each day’s tasks, notice where we duplicate efforts and could be more efficient, organize our materials to spend our time well, and use a checklist to help us remember both key steps as well as additional considerations.

As part of our own internal quality improvement efforts in the CHPR Research and Evaluation Unit, we’ve begun to develop SOPs for our day-to-day work. These include such topics as purchasing participant incentives, in-person data collection activities, and participant outreach.

Hot Tips:

• Create Steps – Pull apart a recent project that you completed, jotting down each large ‘step’ from start to finish (e.g., literature review/environmental scan, data collection, IRB application, data analysis, report and presentation generation).
• Focus In – Focus in on each ‘step,’ listing the various sub-steps in doing that one activity. For instance, within data collection, one might need to identify the sample and sampling methodology, develop recruitment tools to reach those identified, develop confirmation materials for those who agree to be part of our evaluations, identify location(s) for focus groups, sign consent forms, and provide incentives.
• Review and Comment – Put together a review team of 2-3 colleagues who can comment on and edit these SOPs before wider dissemination to your working groups of colleagues.
• Allow Iteration – Include a place on each SOP for users to provide comments and suggestions for updates – this process makes SOPs working documents that will allow for continuous quality improvement activities.

Lessons Learned:

• Creating a template can provide a framework for developing an SOP, ensuring that the salient categories are included and the documents are somewhat standardized across a variety of topics.
• Using reviewers who are not familiar with your projects can help to ensure your SOPs are clear and comprehensive. They can identify missing information or unclear steps.
• Each SOP should be about a very focused activity – this keeps most of them to a one-page maximum in length and thus easy to use/implement.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.