I am Maria Whitsett, a consultant with Moak, Casey and Associates. We specialize in school finance and public education accountability. I also have worked in state agencies, a local school district, and regional R & D and technical assistance centers. My “tips” are old-fashioned!
Lesson Learned: Public sector evaluators need to streamline their own work and improve working conditions for program managers and staff before evaluation is seen as valuable. Stakeholders’ perennial question of “What’s in it for me?” needs a better answer than “Satisfying {district/state/federal} requirements while determining program impact.”
Hot Tip #1. Pay attention to the authorizing environment. Who commissioned or is paying for this evaluation? What must happen for each to believe the mission was accomplished?
Hot Tip #2. Do not scrimp on homework. That includes understanding program purpose(s) and how/why activities are expected to accomplish them. It also includes efficiency in accessing or gathering data necessary for the evaluation. Are there extant data appropriate for this evaluation? Are there existing instruments that can be adjusted or modified to meet the needs of this evaluation? Prevent staff from experiencing onerous additions to their workloads, perceived as unrelated to the “real” work. What quality checks can you implement up front to assure data integrity from start to finish? Re-dos of any kind may carry steep costs that aren’t necessarily reflected in dollars, like damage to trust, credibility, and working relationships.
Hot Tip #3. Communicate with, and learn from, players representing all levels of the program being evaluated. It’s one thing to “see” a program plan and another to understand its day-to-day realities. Such conversations often lead to insights that, ultimately, help identify efficiencies which improve the working conditions for those doing the program. What can staff stop doing without compromising program quality or compliance? Quantify associated savings or improved productivity when possible.
Hot Tip #4. Understand that your role in the organization may end when the requirement ends. There are times when a limited engagement is appropriate, but separation can also represent a lost investment for an organization. This leads to the next tip.
Hot Tip #5. It is extremely important to remember that how you leave an evaluation or an organization is every bit as important as how you entered. Was your work documented sufficiently for someone else to pick up on it? Did you thank players at all levels of the program, both during the evaluation and at its conclusion, for sharing their time and effort with you? Did you establish “next steps” for program providers to consider, even in the absence of continued funding? There is simply no substitution for honoring the organizational and other contributions of individuals responsible for program delivery.
The American Evaluation Association is celebrating Government Evaluation Week with our colleagues in the Government Evaluation AEA Topical Interest Group. The contributions all this week to aea365 come from our GOVT TIG members and you may wish to consider subscribing to our weekly headlines and resources list where we’ll be highlighting Government-focused evaluation resources. You can also learn more from the GOVT TIG via their many sessions at Evaluation 2010 this November in San Antonio.