I am Joy Kaufman, Professor, Yale School of Medicine, a faculty at YaleEVAL, an evaluation group of The Consultation Center at Yale, where I am Deputy Director.
Over the past 20 years I have partnered with recipients of care in evaluating the services they receive with the goal of reducing the power imbalances that can result in the evaluation and design of programs. Although this work has been primarily focused on publicly funded behavioral health services for adults or youth, the methodology could be employed in the evaluation of any program. In our work, consumer-researchers have been trained in all aspects of focus group evaluation — developing the focus group protocols, conducting the groups, coding and analyzing data, and feeding results back to program leadership and policymakers. A key component of this work is that it is completed by individuals who directly receive care. One group has been engaged in this work for eight years and in a few months, we will begin partnering with youth receiving behavioral health care to evaluate the services they receive. A profound outcome of this process is that it has given voice to the needs and desires of the individuals receiving care — people who because of their poverty, limited educational opportunities, and behavioral health difficulties are not consulted when services are developed to meet their needs. Including these voices has resulted in more equitable delivery of care that better meets the needs identified by service recipients.
Lessons Learned: The lived experience of the consumer researchers provides a more contextualized perspective which results in a more complete and nuanced understanding of the data and decreases the potential to perpetuate historical injustices or inequities.
Across each of these evaluations, the data collected by the consumer research team was seen as more authentic and resulted in greater use leading to provision of care that is based on the needs and desires identified by the service recipients.
While a community-based participatory approach to evaluation requires more time, additional resources and a willingness of the evaluator to give up control of the process in order to authentically partner with the consumer researchers, the increased validity, value and usefulness of the results for the program far outweigh the difficulties as the recommendations to program leadership is driven by the work of the consumer researchers elevating the voices of their peers.
Rad Resources: There are several additional resources about this work. (1)The Patient-Centered Outcomes Research Institute provides resources, including funding to further the engagement of consumers in evaluation of health services. Also, below my colleagues and I describe the participatory processes involved in working with consumer researchers in: (2) a first-person account of a CBPR evaluation process that includes perspectives from four stakeholder groups, and (3) a paper on our process to train community members to engage in this work. (4) The Focus Group Kit by Morgan & Krueger includes a helpful volume on training community members to conduct focus groups.
The American Evaluation Association is hosting Promoting Equity Through Participatory Evaluation Approaches Week. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.
This approach is definitely novel and very different than the usual approach that evaluators take when looking at programs. The participants in the program who are receiving care would have intimate knowledge of what works best and how to best implement the program to get the best results. The ability of the evaluator to put their ego aside and listen as opposed to complete control of the program evaluation is admirable and will lead to a more honest evaluation that will help the program be more effective. I feel this is an approach that should be more broadly taken by program evaluators in every field.