BH TIG Week: A multisystem evaluation: Assessing a mental health court in Ethiopia by Faisal Islam

I am Faisal Islam, Evaluation Specialist at Ontario Shores Centre for Mental Health Sciences in Whitby, Canada. Recently, I conducted an evaluation of a mental health court in Ethiopia. First of its kind in Ethiopia, the mental health court was established in collaboration with a number of partners from Ethiopia and Canada. As you can imagine, the evaluation not only dealt with systems and cultures in the two countries, but also with a variety of stakeholders: judges, psychiatrists, correction centre’s staff, police, social workers, and of course, persons with mental health and legal challenges and/or their families.

Hot Tips:

Active awareness of self and others can help the evaluator work for a process that has a fair representation of multiple systems and diverse stakeholders. Overlooking some stakeholders and overly recognizing others can jeopardize the good work that evaluation can bring. Remember: An evaluator is steering the process of evaluation. S/he must be cognizant of power dynamics, organizational cultures and competing interests and use her/his position to assure inclusion.

Openness and willingness to reach-out, negotiate, surrender and adapt can help the evaluator to work for an evaluation framework that has a buy-in. Start early and create appropriate spaces to obtain feedback on the evaluation process. Phone calls, emails, and physical interactions with follow ups can matter in creating shared understanding, building trust and seeking collaborations/ engagements.

Understand inter-relationships and be flexible and respectful in methods of engagement and data collection. Judges, psychiatrists, police, families, and social workers all represent different systems and thus have different priorities and functioning in the mental health court. Yet, the court connects them to perform.  Use system approaches to understand inter-relationships between systems and build evaluation around those relationships.

 Rad Resources:

I found the 2004 special issue of New Directions in Evaluation In Search of Cultural Competence in Evaluation: Toward Principles and Practices extremely helpful in conceptualizing participation and inclusiveness of cross-systems in evaluation.

Bob Williams’ and Iraj Imam’s System Concept in Evaluation: An Expert Anthology is a good resource to understand how systems thinking can be applied in evaluations.

The American Evaluation Association is celebrating Behavioral Health (BH) TIG Week with our colleagues in Behavioral Health Topical Interest Group. The contributions all this week to aea365 come from our BH TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.