We are Joy Kaufman, Associate Professor at Yale University School of Medicine and Director of Program and Service System Evaluation and Evaluation Research and Andrew Case, Assistant Professor or Psychology at the University of North Carolina Charlotte. We are pleased that the Evaluation Use TIG asked us to share work we have done in engaging consumers of mental health services in the evaluation process.
With a primary goal of better understanding consumer perspectives of receiving services at the Connecticut Mental Health Center, four consumer researchers were recruited from the clients served at the Center and trained in all aspects of focus group evaluation. The most salient aspect of this evaluation is the fact that it was developed, implemented and reported by consumers who receive services within the mental health center. Over the past 4 years this team has provided feedback regarding many aspects of care at the Center and their recommendations serve as a blueprint for Center administrators to use in improving the care environment. Perhaps one of the most important outcomes is that this consumer driven evaluation process is now part of how things are done at the mental health center.
Lessons Learned:
Having consumers of behavioral health services evaluate and report their results to the center where they receive care was profound. In our experience as professional evaluators leadership and front line staff, while interested in the results of an evaluation, are often passive recipients of the information. That was not the case in this evaluation, the professionals listened and immediately began reviewing ways to enhance the care experience for consumers.
Having peers lead the evaluation process led service recipients to feel that their voices were heard, a phenomena that consumers of publically behavioral health services do not often experience.
The Center leadership and clinical supervisors reported that the evaluation had added legitimacy and authenticity because of the central role of the consumer researchers.
As evaluators we have learned that while true partnership with service recipients may take more time, the results of the evaluation have increased validity, value and usefulness to the program.
Rad Resources: Patient-Centered Outcomes Research Institute provides resources, including funding to further the engagement of consumers in evaluation of health services.
A first person account of the evaluation process highlighted above was conducted and published in the American Journal of Community Psychology. This paper includes accounts from four stakeholder groups regarding how the project was perceived by stakeholders at the mental health center and the impact of this project on the care environment.
The Focus Group Kit (Morgan & Krueger 1997, Sage Publications) includes a very helpful volume on including community members in focus groups.
The American Evaluation Association is celebrating Evaluation Use (Eval Use) Topical Interest Group Week. The contributions all this week to aea365 come from our Eval Use TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hello Sheila,
Thank you for sharing your post here on AEA363. It was a delight to read and I appreciate your reflection on the impact of the partnership with consumers of the program. I myself have only touched the surface on what evaluation means and how impactful a well developed evaluation can be on a program and your post has added to my inquiry towards getting a deeper understanding of evaluation.
I have always seen consumers and participant’s as the number one priority when conducting any kind of service or program. Your post expresses that not only is their participation important for the success of the program but how we as evaluator can use their first hand experience to enhance the use of our evaluation. You mention how unfortunatly information delivered by professional evaluators are often passive recipients of the information but once consumers delivered the evaluation the information was immediately reviewed and ways of improved began. For me this is not surprising to hear but I do appreciate the evidence that is provided to support this theory. It is true that this relationship between providers and consumers can be challenging, but I do believe it is essential for the consumers to feel as though their voices are being hear. This will only enhance the experience for the consumers.
Since I work within the educational field, specifically within the high school setting, I wonder how to properly encourage and support students to become involved in evaluation. Through the resource you have provided, I have began to brainstorm ideas to bridge the gap but still struggle on way to ensure the evaluation will be impactful for our programming.
Thank you for sharing,
Teaghan