Conducting Evaluation in Academic Medical Centers by Anne Vo and Jacob Schreiber

Hi there! We’re Anne Vo, Ph.D., Director of the Keck Evaluation, Institutional Reporting, and Assessment (KEIRA) Office at the Keck School of Medicine of USC, and Jacob Schreiber, Evaluation Assistant at KEIRA. Today, we offer reflections on what we’ve learned about conducting evaluation within an academic medical center—an environment that offers rich opportunities to observe, conduct, and understand evaluation practice and policy.

Hot Tip #1: Standards Rule Healthcare, Medicine, and Medical Education

Medicine is a highly regulated field. Broad swaths of stakeholders—clinicians, clinical educators, school and hospital administrators—rely on standards to inform decision-making and drive practice. As such, systematic evaluation often manifests as high turn-around monitoring of easily quantifiable outcomes (e.g., student academic performance, residency program match rates, etc.). Successfully “chasing numbers” enables organizations such as academic medical centers to communicate that standards of care and teaching are being met. Because standards offer a common language that stakeholders can use to think through pressing issues of the day, they also become the go-to frame of reference for decision-makers throughout the organization.

Rad Resource:

Hot Tip #2: Everything is “Evaluated,” Everyone is an “Evaluator”

Because standards drive practice in Medicine, evaluation could become a decentralized activity. Aspects of evaluative practice—from question formulation, to data collection, monitoring, analysis, and synthesis—can often be divided among various stakeholder groups across an organization. This cascaded evaluation model emphasizes “local expertise” and echoes “team values” to which healthcare teams aspire. It is reminiscent of development evaluations that organizations such as UNICEF and the UNDP strongly support. And, in the medical context, it is a model that tends to immerse clinical experts in monitoring processes and largely distances them from actual evaluation.

Rad Resource:

Hot Tip #3: Democratic Decision Making is a Core Value

Decisions about how medical education is done are often made through committees and guided by accreditation standards; specifically, LCME Standard 1 on Mission, Planning, Organization, and Integrity (see above link) and Standard 2 on Leadership and Administration. Academic and administrative committees oversee and monitor the quality of Undergraduate Medical Education (what we know as the first four years of medical school). Many of the same stakeholders serve across committees as well as the sub-committees and work groups within each. For evaluation to be meaningful, expect to have many of the same conversations with the same people on different levels. Most importantly, know what each committee’s charge is; its membership; and members’ roles and stances on issues that are up for discussion.

Rad Resource:

Alkin, M.C. and Vo, A.T. (2017). What Is the Organizational, Community, and Political Context of the Program? (pp. 77-87). In Evaluation Essentials: From A to Z (2nd Edition). New York, NY: Guilford Press.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

4 thoughts on “Conducting Evaluation in Academic Medical Centers by Anne Vo and Jacob Schreiber”

  1. Pingback: HPEER TIG Week: Strategies for getting started with evaluation in HPE by Donna Lancianese · AEA365

  2. Anne and Jacob,

    Thank you for an excellent article. I am currently enrolled in my MEd as a nurse and found your article so relevant and it really helped me apply my currently learning to a health care model and understand. I also wanted to say, I didn’t really understand what a decentralized model was prior to reading the article. With a combination of your explanation of why it is important to clinicians and in healthcare and the link and full resource – outlining a decentralized model, I am thankful to have seen all of these resources to further my own understanding.

    My question for you is, after reading the model and your insightful post – are you able to offer any first hand opinion on how a decentralized model really functions? I would anticipate that there would have to be strong communication and trust among teams for it to really be successful. As we have been focusing on organizational lenses, I also anticipate that your organization would have to have specifically solid political lens qualities, such as well established power structures so each member did not feel they were “over stepping boundaries” and at the same time understood their role and responsibility. Is this the case? Do you have any take home messages on seeing this structure in action?

    Thank you

  3. Hi, Jenny. It helps to be reminded that our stakeholders have “evaluation capital” of their own that’s sometimes overlooked. Thanks for highlighting that point!

  4. I love your principle of Everything is “Evaluated,” Everyone is an “Evaluator.” I have found that to be a successful strategy with community work and Collective Impact evaluations as well. It increases engagement and buy in immensely.

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.