Greetings! We are Rakesh Mohan, Director, and Lance McCleve, Principal Evaluator, of the Idaho legislature’s Office of Performance Evaluations. Recognizing the ongoing debate about evaluator credentialing/certification, here we discuss a study that demonstrates the value of credentials in evaluations but not necessarily credentialed evaluators.
As the director, I manage high stakes evaluations. A poorly conducted evaluation could end my career and damage my office’s hard-earned credibility. For me, managing evaluations involves assembling the best team with the resources given.
The best team collectively has the total knowledge necessary to conduct and disseminate a credible evaluation. It includes evaluative thinking, ability to conceptualize complex issues, subject matter expertise, project management experience, research and analytical skills, political savvy, knowledge of evaluation standards, and communication skills. Depending on the type of evaluation, the team may benefit from having certain credentialed members.
In 2008 the Idaho governor proposed a fuel tax to increase highway funding, but many legislators were not interested in raising taxes for a department that they thought was poorly managed. Through a resolution, the legislature asked us to evaluate the transportation department – a highly political assignment. By accepting this assignment, I took the biggest professional risk ever – I had no knowledge of transportation or engineering, the study had a big scope, and the report was due in six months.
I contracted with a team of 11 consultants who brought expertise in transportation, engineering, construction, systems analysis, procurement and contract administration, capital project management, performance auditing, and performance measurement. This team included three licensed professional engineers and a certified project management professional. Additionally, I hired one consultant for financial analysis and two to provide quality assurance and help me manage the politics involved. Interestingly, no one on the team had the title of evaluator; the closest title was management auditor.
The evaluation was a success—on time and under budget! We identified $30.6 million in potential savings the first five years and an additional $6.6 million annually thereafter contingent on implementation of our recommendations.
The governor issued two executive orders: one required the department to implement our recommendations and the other established a task force to identify alternative sources of highway funding. The legislature passed bills raising $27.7 million for highways and appropriated about $10 million to implement the recommended asset management systems.
Stakeholders found the evaluation useful because all parties saw it as accurate, thorough, and unbiased. They knew that I had hired highly qualified consultants to work on the evaluation. Clearly credentials have a place in evaluation but whether evaluators should be credentialed is not so clear. Hence, the discussion about the value of credentialing evaluators should continue.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to firstname.lastname@example.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.