QUAL Eval Week: Laurie Stevahn and Jean King on Essential Competencies for Effective Qualitative Evaluators

Hello everyone—Laurie Stevahn (Seattle University) and Jean King (University of Minnesota) here—continuing to grapple with issues relevant to program evaluator competencies (whether essential sets exist) and usefulness (if enhanced practice results). In fact, for over a decade we have been working on a formal set of evaluator competencies, trying to answer the daunting question of what knowledge, skills, and dispositions distinguish the practice of professional program evaluators. Applying what we’ve learned to the work of “qualitative evaluators” didn’t quite make sense because an evaluator is neither qualitative nor quantitative. As we learned in Evaluation 101, methods come second, once we’ve established an evaluation’s purpose and overarching questions; methods do not guide evaluations. So a competent qualitative evaluator is first and foremost a competent evaluator. But what is a competent evaluator?

Rad Resources: Reading through sets of competencies—there is a growing number of them around the world—can be a helpful form of reflection. We synthesized four competency taxonomies:

1. Essential Competencies for Program Evaluators
2. Competencies for Canadian Evaluation Practice
3. International Board of Standards for Training, Performance, and Instruction (ibstpi) Evaluator Competencies
4. Professional Competencies of Qualitative Research Consultants

Lesson Learned: Thankfully, there was overlap across the domain sets. We can say with considerable confidence that a competent evaluator demonstrates competencies in five areas:

1. Professional—acts ethically/reflectively and enhances/advances professional practice.
2. Technical—applies appropriate methodology.
3. Situational—considers/analyzes context successfully.
4. Management—conducts/manages projects skillfully.
5. Interpersonal—interacts/communicates effectively and respectfully.

Lesson Learned: What distinguishes a competent qualitative evaluator? An enduring commitment to the qualitative paradigm. Qualitative evaluators understand and intentionally use the qualitative paradigm, choosing projects with questions that require answers from qualitative data. They need technical methodological expertise related to collecting, recording, and analyzing qualitative data.

Hot Tip: When using qualitative methods, focus on developing a special “sixth sense” to ensure a high-quality process and outcomes for qualitative studies. This is your ability to interact skillfully with a wide range of others throughout an evaluation to produce trustworthy and meaningful results. It involves interpersonal skills on steroids. A competent qualitative evaluator has to be attuned to social situations and skillfully interact with people in authentic ways from start to finish, knowing quickly when things are tanking.

Hot Tip: In the end, highly specialized sets of competencies unique to a particular evaluator role are less important than your commitment to engaging in ongoing reflection, self-assessment, and collaborative conversation about what effectiveness means in particular conditions and circumstances.

Rad Resource: Stevahn, L. and King, J. A. (2014). What does it take to be an effective qualitative evaluator? Essential Competencies. In Goodyear, L., Jewiss, J., Usinger, J., & Barela, E. (Eds.), Qualitative inquiry in evaluation: From theory to practice. Jossey-Bass, pp. 139-166.

The American Evaluation Association is celebrating Qualitative Evaluation Week. The contributions all this week to aea365 come from evaluators who do qualitative evaluation. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.