Hi, I’m Joe Bauer, the Director of Survey Research & Evaluation in the Statistics & Evaluation Center (SEC) at the American Cancer Society (ACS) in Atlanta, Georgia. I have been working as an internal evaluator at the ACS for almost nine years, in a very challenging, but very rewarding position.
Lesson Learned: Evaluation is always political and you must be aware of those cultural dynamics that are part of every environment. I came to the American Cancer Society to have an impact at a national level. I had envisioned evaluation (and still do) as a means to systematically improve programs to improve the lives of cancer patients.
In the beginning, many were not ‘believers’ in evaluation. The perception was that evaluation could only lead to finding things that were wrong or that were not working – and that this might lead to politically problematic situations. We needed to navigate the cultural mine fields, even as we were acting as change agents. Over time, our Center worked hard to build a sense of trust. As internal evaluators, one must always be aware that we are being judged, as to how nice you are playing in the sandbox, even as we strive and push for higher quality, better data, and better study designs. Evaluators ask the tough questions – which at times cause ‘friction’. However, an internal evaluator must have a comfort level and the confidence with taking that role of asking the tough questions, which can be lonely.
Hot Tips: As an internal evaluator, one must be willing to ‘stay the course’ and ‘weather the storms’ and to never compromise on your values. This is crucially important – because you always need to do the right thing. This does not mean you end up winning all these ‘battles’, because ultimately, you can and are over-ruled on many issues. However, you must keep your integrity – because that is something you need to own throughout your career. That is also what builds trust and credibility.
Rad Resources: The American Evaluation Association’s Guiding Principles for Evaluators http://www.eval.org/p/cm/ld/fid=51 – which are intended to guide the professional practice for evaluators and inform evaluation clients and the general public about the principles they can expect to be upheld by professional evaluators.
The Official Dilbert Website with Scott Adams http://www.dilbert.com/ – where there are many ‘real world’ examples of the cultural dynamics that occur in the world of work and the often absurd scenarios and dynamics that play themselves out. As an evaluator – you will not only need to have a good skill set and work hard at keeping your values and integrity – you will need to have a sense of humor and keep your perspective.
The American Evaluation Association is celebrating Organizational Learning and Evaluation Capacity Building (OL-ECB) Topical Interest Group Week. The contributions all this week to aea365 come from our OL-ECB TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Pingback: “The evaluator’s coming to town!” | Perspectives in Development and Evaluation
Hi Joe,
I fully agree with you. My experiences are similar. As Evaluators sometimes we hear, see and experience things that both our moral and evaluation compass tells to alert and escalate. This often occurs in an environment which is mired by politics.
Since my introduction to the organisation I am working at, I have had to define what evaluation is and isn’t, conduct it in an ever changing organisational dynamic, and think of the longevity of the evaluation role in relation to the oragnisation’s future plans.
This often means having to know when the time is right for certain evaluation activities, and being in tune with the organisation’s rhythm. At university one is schooled to think and behave a certain way according to what you have been taught, however, sometimes one needs to apply intuition in ones conduct. To be overly reliant on the rational side of ones brain is also not always good. I am glad I learned this.
My greatest victories have come when I have reduced evaluation anxiety, and presented the big picture to people. Moreover, the feedback I have been getting with regards to evaluation’s role has been good.
I think when one is an internal evaluator, one needs to also envisage what is best for the organisation, but at the same time, not become “married into the programme”. One needs to be like a like an uncle/aunt in a nucleated family that’s living together. You need to know your place, have your say, be a supportive family member, but not to overstep your boundaries.
What are your thoughts on this?
Asgar- How you described this is so very lovely! I hope you wouldn’t mind too much if I use that analogy. I can see how it would be helpful in some situations. I am often needing to describe and re-frame my role in a project.
Joe – I really appreciate your perspective that evalaution is always political and that engaging in an evaluation process can cause “friction”. In fact, more and more I bring this up early in the planning process. I discuss with teams to expect riction/conflict to arise in the process and that it is good, because it signals that we are connecting with what is important to us as individuals and organizations. It is in that space that we really have opportunity to create not just change, but improvement! I am lucky enough to work with a lot of social workers that this strategy has been beneficial, especially when a team brings up their concerns about conflict to me.
In what ways have you leaveraged the positive aspects of conflict with project teams? I would like to have a few more ideas of how to do that in my toolbox!
Hi Lisa,
Sure you may use my analogy. I am sure someone else might have come up with that analogy before me. So if they have, then credit to whoever that was.
I am always glad to hear insights from other evaluators on this topic. I have read on the AEA website about other Evaluators talking about getting involved in programme activities. Some say do not get involved in programme activities. In my current role I have been involved in programme activities, but primarily relating to the selection of participants onto the programme. I facilitated one or two sessions, but for the most part I have been removed. What I have found interesting through my journey in being an internal evaluator, is that one needs to have a sounding board, or write down what your evaluator gut instinct tells you, vs what your programme experience tells you. For me, another thing I do is to surround myself with literature about internal evaluation, meet other NGOs, attend conferences and follow AEA (or chat to practitioners who are part of SAMEA). This helps me place myself and my role as an evaluator in the greater context of a research community, but also allows me to go back to my immediate context and check whether or not whatever I am doing is relevant. That am I being a critical friend, or a supportive relative or thinking like an investor.
My role in my current organisation has evolved. In particular the projects I have been involved in, and the disciplines I have now taken interest in. So for example, I have been tasked with setting up and M&E System and Framework. The context, is a rapidly changing, data hungry, critical, curious yet sustainable and forward thinking organisation. This is an atypical type of situation (or at least that’s what I thought). This stretched my thinking to think about issues relating to Data Science, Business Analysis, Process Engineering, Organisational Development, Project Management and Information Systems. It’s also forced me to ask the so what question in relation to M&E theory and Capacity Building. A further so what question that was asked was: how is this relevant to the core business of this organisation. How do I position/sell/make it palatable to a non-M&E audience, who are committed to the organisation’s vision.
So, this experience has stretched my thinking of what evaluation is and isn’t and what it could be. To be honest, I think 21st century skills for almost all professions require one to have the skill set of an entrepreneur.
The thing that gives me piece of mind is to ask myself: what has changed since I have arrived, that was in my control, and that resulted in a growth opportunity for the organisation. By taking stock of this, I am able to stay some what sane (in the membrane), as opposed to “insane in the brain” (Cypress Hill).