The Value of an Independent Evaluation Advisory Group for Developmental Evaluators by Barbara Klugman

Barbara Klugman
Barbara Klugman

Hi, I’m Barbara Klugman. I offer strategy support and conduct evaluations with social justice funders, NGOs, networks and leadership training institutions in South Africa and internationally.  This blog is about the value of establishing an Evaluation Advisory Group when your task requires more skills that you have!

I’m currently the ‘learning and assessment partner’ of Tekano an organisation which runs the Atlantic Fellows Programme for Health Equity South Africa.  I have found this experience challenging, not least because it’s my first experience in long term developmental evaluation, because I have multiple accountabilities – to funder, Tekano board, and staff, as well as to the public good given the country’s desperate need for a strong fellowship programme on social determinants of health; and because my primary expertise is in strategizing and evaluating social justice advocacy initiatives rather than leadership development. Finally, I am a white older woman playing the role of ‘critical friend’ to an initiative whose staff and fellows are mostly black and young and whose advocacy experience is taking place in a vastly different historical moment from when I built such experience.

With these constraints in mind, when I developed my terms of reference, I motivated a budget to allow me to establish an Advisory Group. I sought members to fill three gaps in my expertise (that I was aware of) – an organizational psychologist whose expertise traverses organisational behaviour and monitoring and evaluation; a fellowship evaluation expert; and an expert in community-building and evaluation in the South African context.

I engaged the group to help me review the vast quantity of data and analysis I had produced in order to hone in on priorities for my final 18 months in this role, and to help me clarify how to manage my layers of accountability. It was fantastic!! The group brought totally independent eyes to bear on my questions. They helped me distinguish the nature and hierarchy of my various accountabilities. They confirmed my own conclusions on achievements and challenges thus far while hauling me out of the data, insights and relationships I was buried in.  This allowed me to hone in on powerful questions and ways of asking them, for the second half of my tenure in this role.

I loved every minute of the experience even while it was exhausting and challenging. I would encourage you to build in space for this kind of independent support if you’re conducting developmental evaluations.

Hot tips: You have to be a fairly confident person to open yourself up to criticism. Choose people who are ethical and don’t have oversized egos!

Lesson Learned:

  • Building a formal advisory group into evaluation budget allows you to identify exactly
  • what skills might complement your own, and to compensate people appropriately  for their time
  • My huge thanks to Drs. Jane Reisman, Mark Abrahams and Suki Goodman for their efforts on my advisory group.

1 thought on “The Value of an Independent Evaluation Advisory Group for Developmental Evaluators by Barbara Klugman”

  1. Hi Barbara,
    I found your article inspiring and interesting on many levels. First of all, what an amazing experience to be in South Africa working on this type of project – good for you! In reading how you have established your evaluation ‘team’ complete with individuals who, as you put it, can ‘fill the gaps in areas of expertise’ where you feel necessary,certainly speaks to your capability as an evaluator. I am currently enrolled in a graduate program at Queens University in Ontario, Canada, where the discussion of required skill sets for evaluators came up. One of the ‘dilemmas’ facing professional evaluators is that they are capable, competent and qualified to gather, analyze and report data, but may lack the skills and/or knowledge base to make recommendations for improvements/changes to programs. it’s great to see the way you have worked to combat this with the establishment of your advisory team. Do you think establishing such teams when engaging in evaluative processes on a regular basis could be beneficial to the quality of evaluations conducted?

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.