We are David J. Bernstein, a Senior Study Director with Westat, founding chair of the AEA Government Evaluation Topical Interest Group, and President-elect of Washington Evaluators, the DC area affiliate of AEA, and Kathy Newcomer, Director of the Trachtenberg School of Public Policy and Administration at George Washington University, a former AEA Board Member, and a Past President of Washington Evaluators. We both have a long-standing interest in ways to improve the way the U.S. Federal Government contracts for evaluation services.
Problem: The vast majority of United States Federal government evaluations are conducted by contractors, but effective contracting is rarely examined. Government evaluation is not rocket science, but it is complicated.
A. Procurement regulations are detailed, and maybe outdated.
B. Agency practices differ across the Federal government.
C. There appears to be a lack of research focused on contracting for Federal evaluation work (although there are GAO and other studies on Federal contracting).
Solution: At the 2014 AEA Conference, a panel of government evaluators, contractors, and academics addressed 5 questions related to evaluation contracting and how it can be done more effectively. At a July 2015 Washington Evaluators Brown Bag, we presented a summary of the AEA session and asked the audience for opinions and examples on the 5 questions:
- Name one legal and/or regulatory obstacle that can affect the quality of contracted evaluations. Potential solutions?
- Do Requests for Expressions of Interest and question and answer processes improve the quality of evaluation Requests for Proposals (RFPs)?
- How do government estimates of level of effort (or lack thereof) and time frames influence evaluation budgets and the conduct of evaluations?
- How do contractors decide to bid or not? Do certain practices discourage bidding?
- What are the pros and cons of performance-based contracting? Is it possible or desirable for contracting evaluation services?
Hot Tip: Stage evaluation study timelines. During Year 1, focus on evaluation planning: conduct evaluability assessment; create an evaluation plan; develop data collection forms; and prepare for/conduct OMB review. In the outyears, collect and analyze data; build in time for processor implementation reviews to assess program context; and allow time needed for interventions to produce/demonstrate intended outcomes.
Rad Resource: The PowerPoint Presentation summarizing the AEA session is available on the Washington Evaluators website.
Interested in government evaluation contracting? Look for a session on Exemplary Practices in Contracting for Government Evaluation at Evaluation 2015 in Chicago, IL.
The American Evaluation Association is celebrating Washington Evaluators (WE) Affiliate Week. The contributions all this week to aea365 come from WE Affiliate members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.