BH TIG Week: Some Reflections on Requirements for the Use of Evidence-based Practices by David Shern

Greetings. I’m David Shern, the Senior Public Health Advisor for the National Association of State Mental Health Program Directors and Chair of the science committee for the Campaign for Trauma Informed Policy and Practice (CTIPP). Last year, CTIPP tracked the inclusion of trauma informed practices in the opioid bill passed by Congress that included reference to the use of evidence-based practices (EBPs) as a requirement for federal support. The potential requirement that EBPs be used in addressing trauma related opioid interventions prompted lively discussion among members of CTIPP’s science committee.

Lesson Learned: Although CTIPP certainly believes that we should provide interventions and services that have a strong evidence base, we think that it is important to be aware of some of the limitations associated with the strict adherence to EBP requirements. When rating the strength of evidence, randomized controlled trials (RCTs) are typically given the greatest weight. These trials test for mean differences between intervention and non-intervention (often treatment as usual) groups. While this demonstrates that, on average, intervention group participants benefit more from the intervention than persons in the control condition, RCTs often do not explore subgroups in both conditions that either do or do not respond to the intervention. In the era of personalized medicine, average between group differences don’t tell the full story. Furthermore, RCTs frequently lack sufficient numbers of individuals from unique populations to understand the effects on these groups and/or are not conducted in a variety of settings to adequately assess the effectiveness of the interventions for differing groups in differing settings. Such concerns give rise to issues of cultural appropriateness as well as individual preferences. Given the increased mandate to implement EBPs, evaluators need not only assess their effectiveness but should also consider if the intervention is meeting individuals’ needs and preferences as well as whether it is compatible with the skills and preferences of the service providers who will deliver the EBP.

In response to these concerns, the Campaign for Trauma Informed Policy and Practice (CTIPP) has adopted a policy regarding requirements for the use of EBPs. The policy calls for the use of the best available evidence but also anticipates situations in which EBPs are not relevant for a particular application. It calls for a process through which programs can appeal the use of prescribed EBPs with the proviso that they evaluate their adaptations or novel approaches using the most rigorous design possible and that resources be provided for this evaluation. The CTIPP Position on Evidence-Supported Practices and Policies policy brief can be found at:

http://www.ctipp.org/wp-content/uploads/2019/02/CTIPP_ESP_103118.pdf

The American Evaluation Association is celebrating Behavioral Health (BH) TIG Week with our colleagues in Behavioral Health Topical Interest Group. The contributions all this week to aea365 come from our BH TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.