Rakesh Mohan and Lance McCleve on Why Evaluators Should Respond to the Interests of Sponsors and Stakeholders

Greetings from beautiful Boise, Idaho! We are Rakesh Mohan and Lance McCleve from the Idaho State Legislature’s Office of Performance Evaluations. This post discusses our office’s approach to addressing two seemingly conflicting expectations—responsiveness to and independence from sponsors and stakeholders. Instead of worrying about finding the right balance between the two, our mantra is to maximize both of these expectations.

Lesson Learned: We need more than facts and fancy analyses if we want to increase the use and impact of our evaluations—we have no choice but to be responsive to evaluation sponsors and stakeholders. However, this is never easy; especially when working in a politically charged public policy arena where evaluators are expected to be independent from sponsors and stakeholders.

The 2009 Idaho Transportation Department Performance Audit is probably the most high-stakes study we have ever done. Our success depended on how well we responded to the interests of sponsors and stakeholders. The study was requested by the legislature at a time when political tensions were high between the legislature and the governor on how to fund highway projects. Furthermore, the legislature had little confidence in the department’s abilities to manage those projects.

We worked with legislative leadership in developing the scope and securing $550,000 for the study. We identified Idaho’s governor, lieutenant governor, transportation board, and transportation department as key stakeholders. Other stakeholders included Connecting Idaho Partners (a joint venture of two engineering firms) and local highway districts. Throughout the study, we worked closely with all stakeholders. We also sought technical advice and buy-in from the state’s controller, treasurer, attorney general, executive and legislative budget directors, chief economist, and fiscal officer.

Hot Tips – Strategies for Success: In addition to conducting thorough and accurate analyses, we used the following strategies—grounded in the AEA Guiding Principles, the Program Evaluation Standards, and the Generally Accepted Government Auditing Standards—to maximize both responsiveness and independence.

1. Understand the political context by identifying key stakeholders and knowing their relationships with each other
2. Establish trust with stakeholders by involving them in the evaluation and keeping the evaluation process transparent
3. Disclose all conflicts of interest and avoid even the appearance of any conflicts
4. Develop practical recommendations that are sensitive to socio-political and economic conditions
5. Appreciate the time constraints of the sponsors and stakeholders
6. Prepare a balanced report, both in content and tone
7. Have a clear message and an easy-to-follow report presentation for busy policymakers and the press

These strategies helped us avoid political landmines and produce an evaluation report that was useful to both policymakers and the department. Our 2011 follow-up report discusses the impact of the evaluation.

Want to learn more about the work of the Idaho State Legislature’s Office of Program Evaluation? Considering attending session 377 or session 826 at Evaluation 2011, the American Evaluation Association’s Annual Conference this November. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice.