AHE TIG Week: Pat Barlow and Tiffany Smith on Higher Education Assessment in Graduate Medical Education

Hi, this is Pat and Tiffany. We are doctoral candidates in Evaluation, Statistics, and Measurement who work for the Graduate School of Medicine at the University of Tennessee. We completely redesigned a statistics and epidemiology curriculum where none of the previous instructors had formally outlined what they wanted their residents to learn. We not only created a brand new syllabus with learning objectives, but also taught the courses and assessed baseline knowledge and outcomes.

Ask yourself: as an assessment professional (or course instructor), how many times have you been faced with generating useful assessment data from a vague or altogether absent set of learning goals?

Starting from nothing, we had to find a way to gather useful assessment data through the creation of new instruments. Here are five tips that can be used in any assessment or evaluation where there are vague or unclear learning goals.

Hot Tips:

One: Know Your Situation

• Learning environment
  • What is being taught? (For us, statistics and research methods—not everyone’s idea of exciting)
  • What is the nature of the course? (e.g. required vs. optional)
  • Work environment
    • Do the students have external obligations that need to be considered? (Our case, hospital “on-call” obligations)
    • Population-specific
      • What are the factors associated with your target population? (E.g. age, learning style, background with topic).
      • Availability of resources
        • What are your time, personnel, and financial constraints?

Two: Clarify Your Purpose

• Ask yourself two questions:
  • How will the instructor(s) benefit from the assessment results?
  • How will the students benefit from the assessment results?

Three: Use What You Have

• Play detective, gather the necessary background data
  • Existing content, instructor/staff interviews, direct observation, literature, and/oryour previous experience.
  • It provides three benefits: (1) Shows what instructors think the students are learning; (2) what is actually being taught; and consequently (3) where gaps exist in the curriculum.

Four: Fit the Instrument to Your Purpose, Not the Other Way Around

• Always consider situational factors (tip one), and align assessment strategies to the most efficient method for that situation.

Five: Get Consistent and Critical Feedback

• Assessment development/administration must be viewed as a dynamic and iterative process.
• An instrument is developed or modified, it is tested, the testing generates feedback, the feedback leads to modifications to both assessment and teaching and learning activities.

 

We hope these tips will be helpful for your assessment work; good luck!

Rad Resources: For more information on assessment we strongly recommend the following…

• Assessment Clear and Simple by Linda Suskie (2010)

• Internet Resources for Higher Education Outcomes Assessment from NC State University

• For a copy of this presentation along with other resources check out my SlideShare page

The American Evaluation Association is celebrating Assessment in Higher Education (AHE) TIG Week. The contributions all this week to aea365 come from AHE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.