AEA365 | A Tip-a-Day by and for Evaluators

TAG | evaluator competencies

*AEA365 Curator Note: My apologies to aea365 subscribers! Some of this week’s posts did not go out via email to our subscribers. Please read this post on the site itself (http://aea365.org/blog/), and be sure to scroll back to the first post of this series on December 4 in order to enjoy and learn about the overview and history of the Competency Task force and all of the domains!

This is Hazel Symonette, University of Wisconsin, and Sandra Ayoo, University of Minnesota, members of AEA’s Competencies Task Force.  Today we are presenting the competencies from the fifth and final domain: the 2/24/16 DRAFT Interpersonal Domain. Remember that this is the central question guiding our task force: “What is fundamental to the practice of evaluators?”  Given AEA’s Guiding Principles, it is difficult to imagine a skill set more fundamental to evaluation practice than people skills.  Interpersonal competencies revolve around how we relate to and engage others.  These foundational competencies focus on an evaluator’s interactions with the full spectrum of relevant stakeholders, in particular, culturally and contextually appropriate and responsive communications and social relations.  Such considerations are especially crucial in choosing and activating appropriate protocols for establishing and sustaining rapport, for negotiation, and for conflict transformation.

interpersonal-domain

Ongoing Issues:   Our Task Force continues to work through how to best incorporate culture, social justice and power/privilege considerations.  We are striving to integrate cultural considerations across domains given their centrality to evaluation practice.  Similar to the most recent revision of the Program Evaluation Standards, we have explicitly attended to culture in these domains:  Professional 1.2, Methodology 2.2, Context 3.3, and Interpersonal 5.5.

Social justice considerations are more challenging because some evaluators foreground and embrace these issues while others may question the relevance of including this language.   Such considerations revolve around how evaluators appropriately AND effectively engage their understandings of context and power differences—notably, privileging and marginalizing structures and dynamics.  How do we equitably engage and “distribute” the voice and involvement opportunities, resources, and burdens associated with evaluation processes, protocols, and products?  Our work continues. . .

Get Involved: The Task Force continues to seek input as we update this draft for final review and AEA Board approval and for voting by the AEA membership.  Please share your thoughts now on the AEA homepage (www.eval.org). Your voice matters!

The American Evaluation Association is celebrating AEA’s Competencies Task Force week. The contributions all this week to aea365 come from members of AEA’s Competencies Task Force. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

This is Gail Vallance Barrington, Barrington Research Group, Inc., representing the Canadian Evaluation Society, a member of AEA’s Competencies Task Force. Our Task Force’s charge was to answer one essential question: “What is fundamental to the practice of evaluators?” As an active participant in CES’s Credentialed Evaluator Program, I have been pleased to share our credentialing experiences north of the border as AEA establishes a set of formal evaluator competencies.

The competencies included in the 2/24/16 DRAFT Management Domain focus on the nuts and bolts of conducting an evaluation, such as finding work, budgeting, coordinating resources, and monitoring progress. Also included is timeliness both in communication and in completing an evaluation.

management-domain

Lesson Learned/Issues Raised: Feedback to date has questioned the domain name. Several people noted that, as they are not managers, the heading Management Domain does not seem to apply to them. We are considering changing the name to Project Planning and Management Domain because whether we are lead evaluator or not, we must plan and manage our work efficiently. Let us know if this change makes sense.

Another suggested change relates to the evaluator’s role in working with others: “Coordinates and supervises others to deliver evaluation processes and products effectively.” Again, feedback is welcome.

Get Involved: We need to hear from as many AEA members as possible in the coming weeks. Let us know your thoughts by going to the link on AEA’s homepage (www.eval.org) and contacting us. Also, keep reading aea365 this week for draft competencies from the remaining two domains.

The American Evaluation Association is celebrating AEA’s Competencies Task Force week. The contributions all this week to aea365 come from members of AEA’s Competencies Task Force. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

This is Eric Barela, Salesforce.org, Anna Rodell, Collective Progress, and Susan Tucker, Evaluation & Development Associates, members of AEA’s Competencies Task Force. This week we are presenting the competencies our group has drafted in the past 18 months. One grounding question has guided the work of our task force: “What is fundamental to the practice of evaluators?” An important answer to that question is evaluators’ ability to understand the context in which they practice.

Competencies included in the 2/24/16 DRAFT Context Domain focus on analyzing and attending to the unique interests, issues, and circumstances surrounding any given evaluation. This includes attention to the evaluation’s setting, the details of the evaluand itself, the concerns of numerous stakeholders, the potential users and uses of the evaluation, and the evaluation’s broader social context.

context-domain

Lesson Learned: One of the questions we asked in drafting these competencies was what exactly we meant by a program’s context. Draft competency 3.3 includes examples of contextual features. These may involve the geography, history, and politics related to the evaluand and the values and culture of program recipients.  There are also questions surrounding power and privilege, potentially related to who is in charge and whose voices are present or absent. After extensive discussion, we learned that it is also important that context incorporate broader societal concerns that inevitably affect how programs operate. Our solution was to target both local contextual features and broader social concerns.  In writing this draft, we worked to address context at these two levels and now seek feedback about whether this makes sense as we move to finalize the draft.

Get Involved: Please help by providing your thoughts on the draft competencies. The AEA website (www.eval.org) has an extremely visible link to the full draft of proposed AEA competencies, and we hope you will send your ideas to us at competencies@eval.org. Also, check tomorrow’s aea365 for draft competencies from the one remaining domain (Interpersonal).

The American Evaluation Association is celebrating AEA’s Competencies Task Force week. The contributions all this week to aea365 come from members of AEA’s Competencies Task Force. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

This is Dale Berger, Claremont Graduate University, Robin Miller, Michigan State University, and Michelle Gensinger, University of Minnesota, members of AEA’s Competencies Task Force. The competencies included in this 2/24/16 DRAFT Methodology Domain are those that focus on the methodological knowledge and skills fundamental to evaluation practice.

method-domain

Unresolved Issues:  In developing the methodology competencies, our Task Force identified three underlying tensions where we ask for your help in shaping the final content.

  1. Its very name suggests that evaluation is an act of judgment, and yet, for many practitioners, evaluation practice includes many other activities. In competency 2.12 we included the phrase “when appropriateto acknowledge the importance of evaluative judgment while not suggesting that every evaluation addresses it.
  2. We included competencies for two activities that not all evaluators may engage in: literature reviews, and meta-evaluations. Our thought was to include the first because evaluators should ground their work in what is known from existing research and the second because it is one of the domains of the Program Evaluation Standards.
  3. There is no competency related to developing recommendations. The question of whether or not to include recommendations reflects another long-standing tension in the field.

Get Involved: As noted, the task force is serious about getting input from AEA’s broad membership to be sure that we consider multiple perspectives as we prepare the final version for a vote by AEA members. Let us know your thoughts on the AEA homepage (www.eval.org). Also, keep reading aea365 this week for draft competencies from the remaining three domains.

The American Evaluation Association is celebrating AEA’s Competencies Task Force week. The contributions all this week to aea365 come from members of AEA’s Competencies Task Force. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

This is Laurie Stevahn, Seattle University, Donna Podems, OtherWISE: Research and Evaluation, and Nicole Galport, Claremont Graduate University, members of AEA’s Evaluator Competencies Task Force. The work of our task force has been guided by a grounding question: What is fundamental to the practice of evaluators?

Competencies that comprise the 2/24/16 DRAFT Professional Domain are those deemed to fundamentally ground the professionalism of evaluator practice and distinguish evaluators from others who engage in professional inquiry.  These competencies include knowing and adhering to standards, guidelines, and principles adopted by AEA, our professional association (i.e., the Joint Committee Standards, AEA Guiding Principles, AEA Public Statement on Cultural Competence), engaging in ethical and culturally appropriate practice, and being reflective practitioners—knowing oneself as an evaluator, communicating that to others, and striving for continuous growth.

professional-domain

Lesson Learned—and Assistance Sought:  Putting it mildly, wording competencies is a non-trivial challenge! Competency 1.5 borrows the phrase “the general and public welfare” from AEA’s Guiding Principle 5. AEA members approved this wording when the Guiding Principles were developed. We have received conflicting feedback to date about this language. On the one hand, some people question whether such content should even be included as an evaluator competency; on the other hand, some question why we are not using stronger language related to social justice, rather than to “the general and public welfare.” We welcome thoughts on how to include this important, but difficult content in the professional domain.

Get Involved: The task force continues to seek AEA membership input as we progress toward a final revision that, following Board approval, will ultimately be presented to the AEA membership for a vote. Your feedback matters, so make your voice heard by going to the link on the AEA homepage (www.eval.org) and providing your input. Also, check aea365 in the coming days for draft competencies from the remaining four domains.

The American Evaluation Association is celebrating AEA’s Competencies Task Force week. The contributions all this week to aea365 come from members of AEA’s Competencies Task Force. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

This is Jean King, University of Minnesota, chair of AEA’s Competencies Task Force. This week Task Force members are excited to share our progress toward developing a set of AEA evaluator competencies.  To put these efforts in context, a bit of background is in order. In 2015, the AEA Board approved the creation of a task force to explore and refine a unified set of evaluator competencies as a next step in AEA’s continuing development.  AEA members have previously endorsed three documents contributing to the professionalization of our field (Rad Resources all):

  • The Joint Committee’s Program Evaluation Standards, currently in their 3rd edition (2011, http://www.jcsee.org/program-evaluation-standards-statements)
  • AEA’s Guiding Principles for Evaluators (revised in 2004, http://eval.org)
  • AEA’s Cultural Competence Statement (2011, http://eval.org)

The proposed competencies will be the fourth such document.

As an initial step, the Task Force reviewed existing sets of general and subject-specific competencies for program, policy, and personnel evaluators to identify foundational competencies necessary to the diverse evaluation practice of AEA members. The resulting crosswalk suggested five broad domains of evaluator competencies: professional, methodology, context, management, and interpersonal.

Who’s on the Task Force? When we were appointed, special care was taken to include as diverse a group as possible, representing numerous segments of AEA’s membership. Here is the list of members (in addition to me):

  • Sandra Ayoo, University of Minnesota
  • Eric Barela, Salesforce.org, San Francisco, CA
  • Dale Berger, Claremont Graduate University
  • Gail Vallance Barrington, Barrington Research Group, representing the Canadian Evaluation Society
  • Nicole Galport, Claremont Graduate University
  • Michelle Gensinger, University of Minnesota
  • Robin Miller, Michigan State University
  • Donna Podems, OtherWISE: Research and Evaluation, Cape Town, South Africa
  • Anna Rodell, Collective Progress, Minneapolis, MN
  • Laurie Stevahn, Seattle University
  • Hazel Symonette, University of Wisconsin
  • Susan Tucker, Evaluation & Development Associates LLC
  • Yuanjing Wilcox, EDucation EValuation EXchange, Golden Valley, MN 

Lesson Learned: AEA is not alone in addressing professionalization. Similar discussions are growing in frequency and intensity around the world. Consider two examples. Our colleagues in the Canadian Evaluation Society offer the Credentialed Evaluator (CE) Program, currently the only formal credentialing available to evaluators anywhere (http://evaluationcanada.ca/ce). The European Evaluation Society and the United Kingdom Evaluation Society have joined forces to develop the Voluntary Evaluator Peer Review process, whereby evaluators will prepare portfolios for review by qualified peers, leading to purposeful professional development (http://www.europeanevaluation.org/events/ees-conferences-and-events/conferences/evalyear-2015x/vepr-project). 

Hot Tip– Get Involved Now: From the beginning, we have sought feedback from AEA’s membership. Please contribute to the discussion by going to the AEA website (www.eval.org) where you will see the link to the draft competencies. Send your thoughts big or small to us at competencies@eval.org.

The American Evaluation Association is celebrating AEA’s Competencies Task Force week. The contributions all this week to aea365 come from members of AEA’s Competencies Task Force. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

I’m Susan Kistler, the American Evaluation Association’s Executive Director, and I contribute each Saturday’s aea365 post. The AEA Board of Directors has adopted a Question of the Quarter to generate discussion with and among members around issue critical to the field.

This quarter’s question is:

  • What are the pros and cons of AEA instituting a certification and/or credentialing program?

Hot Tip: If you are an AEA member, sign up for this month’s Thought Leaders Discussion. François Dumaine and Keiko Kuji-Shikatani helped to shepherd the Canadian Evaluation Society’s Credentialed Evaluator program. They’ll be discussing issues related to evaluator competencies, certification, and credentialing.

Hot Tip: What do you think? Share your thoughts related to the pros and cons of evaluator certification and credentialing via the comments section below. If you are reading this article in email, click on the title to return to the aea365 website and scroll down on this entry to find the comments box.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · · ·

Archives

To top