Hello! I am Scott Swagerty, PhD, Vice President of Budget and Finance for the Arizona Evaluation Network and the Methodologist for the Office of the Arizona Auditor General. I work in the Performance Audit Division which conducts audits to assess the effectiveness and efficiency of Arizona state agencies and programs. I wanted to share some Hot Tips for applying evaluation principles to assess and enhance the effectiveness of state government.
These tips are all based on principles of evaluation that I have adapted to my work in performance auditing of state agencies. However, they can be applied in most evaluation contexts and in my experience help to create a more collaborative and functional relationship between evaluator and client.
- Convince the client that evaluation is useful—unlike in traditional relationships between client and evaluator, when I work with state agencies it is typically not by invitation and our presence can be intrusive. Being prepared to tell and show the client how an evaluation can help them helps to cultivate a strong working relationship with the agencies we audit.
- Rely on the experts—my expertise is in quantitative methodology and research design. Performance auditors’ expertise varies, but does not always coincide with the subject matter we are evaluating. Relying on the agency staff and management to help us understand the subject matter is essential in producing a useful evaluation because they are the ones who understand their processes best and know whether our suggestions for improvement will lead to meaningful change.
- Focus on what can be changed—it is true that in many state agencies there is a shortage of resources that potentially limits the agency’s ability to effectively achieve their mission. However, an evaluation focused on the lack of resources is not useful or actionable because statewide resource allocation is not an agency-level decision. By focusing on evaluating processes or programs as they presently exist, we can suggest changes that improve service delivery to citizens without requiring additional resources.
- Make flexible recommendations for improvement—generally problems or bottlenecks in a process are easily identifiable, but the solution(s) to fix those problems are not so straightforward. Harkening back to the principle of “rely on the experts,” I believe that rather than prescribing a specific solution, it is best when possible to make recommendations that allow the client to design an appropriate solution in conjunction with their management and relevant staff considering the resources available to them. This approach allows for creativity and innovation beyond the program/process being evaluated and invites the client to be more invested in the outcome.
- Keep the discussion on working effectively with clients going at the Arizona Evaluation Network’s 2018 From Learning to Practice: Using Evaluation to Address Social Equity conference taking place this April in Tucson, AZ.
- American Evaluation Association’s Evaluation Roadmap for a More Effective Government
The American Evaluation Association is celebrating Arizona Evaluation Network (AZENet) Affiliate Week. The contributions all this week to aea365 come from our AZENet members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to firstname.lastname@example.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.