GOV TIG Week: Ariana Brooks on When Evaluation Cannot Solve Management Issues

My name is Ariana Brooks and I am the Director of Evaluation, Research and Planning for HeartShare Human Services.

Lesson Learned: When I started as an internal evaluator my supervisor, Stan Capela, stressed to me one main point: evaluation does not solve management problems. My initial reaction was it made sense and remember similar issues discussed in graduate school. I did not fully grasp the meaning until I was performing my job responsibilities. Specifically, each report was producing similar results. At first I was naively shocked at the level of resistance from some managers. We were well versed in Patton’s Utilization Focused approach. So we focused on providing meaningful reports, but there was resistance even though we would repeatedly tell managers the “numbers don’t lie”.

Lesson Learned: As a social psychologist, I reflected on various theories that helped explain their behavior. Of course, people will interpret stimuli based on their own perspective. People are motivated to preserve a positive sense of self and are more resistant to counterattitudinal messages, especially if they are highly invested in the issue (e.g. their job). So it made sense that when an internal audit illustrates program’s deficiencies have more to do with supervision or program administration it can be hard for management to swallow.

Although it is frustrating when management’s resistance to change can reduce the utility of evaluation work, it is fascinating to see how the theories I studied play out in an organization. Borrowing from evaluation and social psychology theories, here are some tips that helped me combat and understand resistance:

• Hot tip: Think about the source of the message, or evaluation results. The source should be respected, seen as having expertise, trusted and viewed as an in-group member (someone also invested in the program or in a similar role).
• Rad Resource: The appreciative inquiry approach to evaluating programs has been met with great success. Managers are more willing to be involved and use evaluation results when they carry a more positive tone. Focusing on management’s strengths to overcome program challenges has proved to be a more useful approach. A great resource online is: http://appreciativeinquiry.case.edu/
• Hot tip: Avoid any language that seems targeted towards certain individuals, roles or positions. Make the responsibility of overcoming challenges a group effort, including the evaluator.
• Hot tip: Take a sign of defensiveness as a positive. Often this is a sign that staff is truly invested in the program and their work. Directing this energy toward more productive means can be a bit of a struggle but be rewarding in the long run.

The American Evaluation Association is celebrating GOV TIG Week with our colleagues in the Government Evaluation AEA Topical Interest Group. The contributions all this week to aea365 come from our GOV TIG members and you can learn more about their work via the Government TIG sessions at AEA’s annual conference. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.