AEA365 | A Tip-a-Day by and for Evaluators

TAG | competencies

Hello! I’m Kathy Newcomer. Serving as president of AEA this year has been an honor and privilege for many reasons. One of them is the opportunity to witness firsthand the incredible commitment and effort so many of our members exert on our behalf!

AEA members serve on TIGs, task forces and working groups and work diligently behind the scenes to move our association and profession forward on a variety of fronts. This year through these  groups our members have made many achievements that benefit us and that I want to acknowledge.

  • Our Evaluation Policy Task Force (EPTF) strategically worked to develop and sustain a coalition of professional associations to provide input to the deliberations of the Commission on Evidence-Based Policy. Our EPTF provided valuable testimony, and our AEA Roadmap was cited multiple times in the final Commission’s report. The recommendations on evidence-building capacity reflected our roadmap, as well as the significant contributions of AEA member and Washington Evaluators president, Nick Hart, who was a key author!
  • Our Race and Class Dialogues series led by Melvin Hall presented valuable forums for discussing how we as professional evaluators can address critical issues facing our society, and due to the dedication and time developed by Melvin and his committee, and funding provided by the Kellogg Foundation, AEA will provide an outstanding training video on this vital topic.
  • Our Competencies Task Force led by Jean King moved toward completion of their multi-year effort to develop and vet a set of evaluator competencies. Members devoted an impressive amount of time conducting focus groups, surveying our membership, and consulting with evaluators globally to ensure our competencies are comprehensive, reliable and valuable.
  • Our Guiding Principles Review Task Force led by former AEA President Beverly Parsons reached out extensively to our membership, including via a survey this fall, to update our association’s guidance to ethical practice.
  • Many members participated in shaping our selection process for a new Executive Director under the leadership of our ED Selection chair and President-elect Leslie Goodyear through contributing valuable guidance on the job description and criteria.
  • Our Membership Engagement Task Force led by Melvin Hall and Robin Miller reviewed AEA records and actions and solicited members’ input to develop a set of actionable recommendations to strengthen our association’s commitment to diversity and inclusive leadership development and membership engagement.
  • Our AEA representative to IOCE Cindy Clapp-Wincek represented us across the world, and led a group of us to participate in an awe-inspiring summit of EVAL partners in Kyrgyzstan.
  • My 2017 Conference Program Committee, comprised of 17 members from 7 countries, worked to develop our themes, recruit speakers and organize a video contest and sessions to ensure our conference provides a memorable learning experience for all.
  • Our network of affiliates led by Leah Neubauer and Allison Titcomb worked to enhance sharing across their organizations and planned their first ever affiliates workshop for Evaluation 2017.
  • Needless to say, we have all benefited immeasurably from the efforts of our TIG leaders who worked long and hard to solicit and vet conference proposals, among other important services they provide to AEA.
  • And 21 working groups comprised of more than 125 AEA members work closely with our Executive Director to conduct essential association business in a variety of areas including elections, awards and international outreach.

My most important role as outgoing president is to bear witness to the achievements of so many of our members who work on our behalf with little recognition other than seeing good work accomplished to move our profession forward. THANK YOU!!!!! We truly appreciate what you have done for us!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · ·

This is John LaVelle, Louisiana State University, and Yuanjing Wilcox, EDucation EValuation EXchange, members of AEA’s Competencies Task Force. Task Force members recently shared the 2/24/16 draft AEA evaluator competencies in five domains: professional, methodology, context, management, and interpersonal. Feedback in coming months will enable us to finalize the set in preparation for two very important engagement activities: (1) a survey of all members to determine the extent to which they agree that these competencies are the right ones for AEA, and (2) a formal vote on the competencies, including a process for their routine revision, thereby making them an official AEA document.

Hot Tip: Keep your eyes open because he Task Force is working on creating professional development materials to enable evaluators, wherever they work, to use the competencies to reflect on their practice and to assess specific needs.  We believe that it is in the reflection process that the explicit value of the competencies will shine as evaluators use them to shape effective practice.  For example:

  • Novice evaluators, those entering the field who want to identify areas of strength and need for development
  • Accidental evaluators, people who may not have formal training, but who are responsible for conducting evaluations
  • Professionals in transition, such as those who may be experts in a particular field, but who want to become competent evaluators in that specific area
  • Experienced professional evaluators, who want to stay abreast of changes in the field’s practice and theory

We envision an individual assessment process similar to that used for the Essential Competencies for Program Evaluators (http://www.cehd.umn.edu/OLPD/MESI/resources/ECPESelfAssessmentInstrument709.pdf) and an interactive process that groups of evaluators (e.g., members of a firm, students in a cohort) could use to customize the competencies to their specific settings.

Lessons Learned: Feedback on the first draft of AEA competencies raised the question of to what extent individual evaluators need to demonstrate each of the competencies because many evaluators work in collaborative groups. We added one competency (Interpersonal Domain 5.7) to address the fact that for many evaluators teamwork skills are essential. We believe that the question of whether the entire set of competencies should apply to individual evaluators versus teams is context-dependent; we invite people to use the competencies as suits their settings and practice.

Rad Resources: If you are interested in a quick orientation to the world of evaluator competencies, consider these three readings:

  • King, J. A., Stevahn, L., Ghere, G., & Minnema, J. (2001). Toward a taxonomy of essential evaluator competencies.  American Journal of Evaluation, 22(2), 229-247.
  • Russ-Eft, D., Bober, M. J., de la Teja, I., Foxon, M. J., & Koszalka, T. A. (2008). Evaluator competencies: Standards for the practice of evaluation in organizations.  San Francisco, CA: Jossey-Bass.
  • Wilcox, Y., & King, J. A. (2014). A professional grounding and history of the development and formal use of evaluator competencies. Canadian Journal of Program Evaluation, 28(3), 1-28.
  • Buchanan, H., & Kuji-Shikatani, K. (2014). Evaluator competencies: The Canadian experience. Canadian Journal of Program Evaluation, 28(3), 29-47.

Hot Tip: See you at #eval17 where we hope to unveil the final draft competencies!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Hello, my name is Diane Rogers and I am a doctoral student in the Interdisciplinary Doctoral Program in Evaluation at Western Michigan University (WMU) and a secondary mathematics teacher. My work in evaluation focuses on education and social justice, so naturally I was excited to have the opportunity to work with and learn from Dr. Rodney Hopson during his visit to WMU.

As a teacher and school improvement chair in an urban school district, I became quickly frustrated with my, and others’, lack of knowledge about using systematic inquiry to make decisions. It seemed as though decisions for school improvement were made for convenience, maintaining the status quo, or because someone had a feeling that something would work. As my school went through the restructuring process, I saw that evaluation could be part of that solution. Yet, in a school full of diverse students and needs, evaluation without cultural competence and adherence to professional standards will not help to ameliorate the complex issues of the educational system. The importance of the concepts of cultural competence and professional standards to our work as evaluators are two of the lessons that Dr. Hopson’s visit highlighted

Lesson Learned: As part of the visiting scholar series at WMU, Dr. Hopson facilitated a discussion with The Evaluation Center staff about the need for and application of AEA’s Statement on Cultural Competence in Evaluation (http://www.eval.org/ccstatement.asp). I left the meeting with a renewed understanding that cultural competency is not a checklist of knowledge and skills that an evaluator can accomplish. Rather cultural competence is a process that requires reflection, learning, dialogue, and more reflection throughout one’s life. For me this process began by exploring my various societal labels and unpacking how those identities impact my worldview, language, and life ways.

Lesson learned: I was privileged to attend a panel discussion of the Program Evaluation Standards (PES) between Dr. Hopson and Dr. Daniel Stufflebeam. (View the video here: http://vimeo.com/33177056) I gained a great deal of insight into the standards from hearing Dr. Stufflebeam’s account of the history of the PES and Joint Committee and Dr. Hopson’s experience in revising the PES. Both scholars communicated the importance of standards to our profession and the need for all evaluators to not only adhere to the standards but also participate in professional conversations about the meaning and application of the standards. It is not enough to own a copy of the PES; you have to read, process, and apply the information throughout your work.

Rad Resource: View a summary of the Program Evaluation Standards here: http://www.eval.org/evaluationdocuments/progeval.html

All this week, we’re highlighting posts from colleagues at Western Michigan University as they reflect on a recent visit from incoming AEA President Rodney Hopson. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · ·

Kia ora and New Year Greetings to my evaluation colleagues.  My name is Kate McKegg. I am an independent evaluation consultant based in New Zealand. My company, The Knowledge Institute Ltd is a member of a professional network – the Kinnect Group. I am a founding member and former Convenor for the Aotearoa New Zealand Evaluation Association (anzea), and I am also a member of the anzea Evaluator Competency Working Group.

As the evaluation profession has grown here in New Zealand, we have been challenged to develop culturally relevant and appropriate conceptions of what quality means; for evaluators, for those who commission evaluation and for the evaluation products of our work.

Like other evaluation associations, we have set about developing ‘evaluator competencies’ that we hope will recognize our cultural context at the same time as guiding and informing sound and ethical evaluation practice.  I’d like to share some things we have learned along the way for others thinking about or already involved in developing evaluator competencies.

Hot Tip – Never underestimate how much relationships matter. Developing evaluator competencies required us to look deeply into what we individually and collectively value about being evaluators, about evaluation, and its consequences.  We found that the values embedded in our diverse relationships with people of all kinds, our colleagues, our communities, evaluation commissioners, were integral to the evaluation competencies we developed.

Rad Resource: The most important resource was the investment of energy and time in finding strong, committed and diverse people who trusted in each other, when the going was good, as well as rough, to lead and do the work.  The journey is certainly not for the faint hearted!

Hot Tip –  Search out the wisdom and experience of others who have trodden the ground before you. Although the New Zealand context is unique, we learned so much by reaching out to others who have already been down this road.  For example, a few of us attended sessions at AEA conferences and talked with representatives from The Canadian Evaluation Society who were in the process of formalizing competencies into their Professional Designations Program.   We spoke with other evaluators such as Jean King and Bob Picciotto, who have been involved in researching evaluator competencies for many years.

Rad Resource: Anzea produced a précis of the published literature and other resources and information and this became a valuable resource for the working group, as well as for others in the association to ‘come up to speed’ with the key issues during the consultation phase.  The Canadian Evaluation Association has also produced useful resources that may be found on their Professional Designations Project Archives Page

· · ·

Archives

To top