I am Michael Quinn Patton of Utilization-Focused Evaluation. Engaging intended users I have to be thoughtful about words, attempting to avoid jargon and words that make people cringe. What those words are can change –and surprise. Take, for instance, the notion of competency.
The profession of evaluation is fairly obsessed with competency: a major factor in use is credibility, which is affected by perceived and actual evaluator competency. The AEA Guiding Principles and Joint Committee Standards highlight evaluator competency. AEA endorsed Cultural Competence in Evaluation in 2011 and general evaluator competencies in 2018. Canada, New Zealand, and Japan have been leaders in developing evaluation competency frameworks. The UN Evaluation Group has competencies for both evaluators and commissioners of evaluation. These frameworks specify competencies in methods, professional practice, interpersonal skills, project management, and context sensitivity.
In working on Blue Marble (global) Evaluation, I adapted the global competencies framework of Columbia Teachers College. Blue Marble evaluators (thinking globally, evaluating globally) should be globally competent. All was going well until I checked back with colleagues involved with the original AEA cultural competence statement, who were doing some rethinking.
The concept of competence comes with deep cultural, imperialist, colonial, and oppressive history. Competency tests have historically been a way of excluding rather than including. Competency tests were administered to Blacks in the South by white electoral judges who decided whether they were competent to vote. Residential schools that took indigenous children from their communities and cut them off from their families, culture, and language proclaimed that they wanted them to become “competent” to succeed in modern society. Throughout the world, training colonized people to become competent was a way of imposing dominant cultural, political, and educational values on oppressed peoples. Competency tests often continue to play that role.
Another concern: competency can constitute a fairly low bar as an entry-level, minimum standards construct. I don’t want a doctor or lawyer who is just competent. Or an evaluator who is just competent. We want people to demonstrate excellence. Criterion of excellence, mutually agreed to, with diverse and contextually appropriate engagement, may be more meaningful and useful than minimum competency standards.
Finally, competency connotes a binary state: competent/incompetent. The concept is static. Get licensed, certified, you are competent. A dynamic view would emphasize ongoing learning and development, not a fixed state of competency.
So, in the end, we are being more circumspect and sensitive about the notion of global competency, the meaning of competency in different contexts for different stakeholders, and placing greater emphasis on having Blue Marble evaluators aspire to being World Savvy, a commitment to ongoing learning about and engagement with global issues, connecting the global to the local, and connecting social justice with environmental justice.
Hot Tip: Be language savvy. Examine old terms in light of new and changed understandings and sensitivities.
Rad Resources:
- Rheingold, H. (1988). They have a word for it: Untranslatable words and phrases. New York: St. Martin’s Press.
- Hopson, R. (Ed.) (2000). How and why language matters in evaluation. New Directions for Evaluation, No, 86 (Summer).
- World Savvy. www.worldsavvy.org
The American Evaluation Association is celebrating A Look at Language Week where a group of Minnesota-based evaluators working in justice and equity spaces contribute articles reflecting on the words we use. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hello,
I am currently completing my Professional Master of Education with a specialization in Aboriginal Education at Queen’s University in Ontario. Right now, I am completing a course called Program Inquiry and Evaluation and I am working on a design for an Aboriginal led social program for youth. I was drawn to your blog post because of the focus on how research is shared and evaluated through an Indigenous perspective.
Even though I am writing you from Canada, our Indigenous population is struggling with the same trauma based issues stemming from colonial violence. As Canada pushes forward towards truth and reconciliation with Indigenous members and communities in Canada, there has been a strong surge of Aboriginal education efforts in the country. As the Truth and Reconciliation Commission revealed the truths of many residential school survivors, many Canadians “woke” up to the history of traumas amongst many aboriginal people, communities, and livelihoods in Canada. It is important for professionals to take the next step within their practice. We need to be aware and work towards an education that is balanced and culturally appropriate to the context of the people we work with.
I think one of the biggest things we can work towards is decolonizing the language and ideas we use when conducting evaluations within Indigenous communities and programs. By this I mean we need to look at the words we use and the meanings behind these words. We need to ask ourselves “is this the right word to use for the idea I am trying to convey, or is there a more suitable term or phrase I can use from the specific Nation’s language that supports the beliefs and values of the Nation I am working with?”.
For example, the Better Evaluation Rainbow Framework (for program evaluation) has seven essential steps to evaluation: manage, define, frame, describe, understand causes, synthesize, report and support use. In the context of the specific Nation, the evaluator would need to decolonize these terms to do a culturally appropriate evaluation.
Thank you for the blog post, I look forward to following more of your work.
Thanks Anna. You’ve deepened the discussion with your experience and insights. MQP
I agree with avoiding the concept of competence or being very circumspect when using it. World Savvy is a great way to add the real meaning, you are looking for in part because since it is a “new” descriptor, it doesn’t carry the baggage I think. Beyond the clearly cultural contexts as described in Dr. Patton’s post, I have found competence to be perceived as fairly subjective because of its historic and current applications. I work in the Federal Government and much talk centers around demonstrated competency but the evaluator is usually a supervisor or a hiring official with a general description serving as the operationalized definition. It is a human who decides and as we all know, there are a multitude of factors at play in any decision. I prefer to refer to experience and education which I believe can be illustrated with more specific examples. My portfolio describes my performance. I am not necessarily referring to a resume or biography because a portfolio could be specific to an area such as “planning” or “group facilitation” Competency as reflected in actual happenings or defined by equating events in someway may help but I prefer not to use the word outside a very narrow context.
Plus, I heartily agree with Dr. Patton’s point about the tendency to equate competency with a minimum standard. When working on process design and goals/objectives in all kinds of planning, competency does set a fairly low bar when used in narrative description or when determining performance levels. Both the word and concept of competence unless clearly operationalized and contextualized adds very little value as a standard. It is easy to describe someone or something’s performance as competent based on current understanding of the term but the more critical questions will always be “so what?”
In my opinion, it is also important to make sure when using this term that you are clear about what you are describing–it is usually applied relating to people–an inspector demonstrates competence but the inspection itself could be poor based on its processes or protocols. Is the inspector incompetent if he/she has had a broken tool that has caused their “counts or hits” to go down? They shouldn’t be IMHO, which is why evaluation of performance often has more substance than a scorecard or similar reporting protocol allows. Personal performance needs to be somewhat clearly separated from process and tool performance. People use “process improvement” methodologies and projects to attempt to mitigate personal performance issues. All of these shades of grey surrounding the word competency and its use make it important to ensure you really are being clear and adding context and not just using a word lazily affiliated with jargon or preconceptions.
That is really the whole issue around language–as evaluators and planners, the words we use should add value and clarity; aiming for that has presented me with more challenges in my career and work than anything else defined in any standard of competency has.
Language is the most powerful tool humans have and how it is used determines the value it adds in any given context. “World savvy” is a much cooler term to use and offers much greater potential for definition and application. If it does nothing else, I bet people are more interested in hearing about what it means and how it applies than any discussion of competency. Sometimes getting engagement is the best way to demonstrate competency. 🙂
I’ve been thinking along a similar vein about our organization which has people with and without visual impairments working together to create distance learning products for people with visual impairments. We are moving more and more toward hiring only those with professional certifications, which tends to leave out blind/visually impaired people. The historic tension between professional associations and disability groups is playing out, but most of our organization is missing it. Most of the top leaders do not have disabilities, and they have not encountered blindness (and other) empowerment messages.
Great example Lydia. Very useful and significant application of the principle in my post. Thanks. MQP