Good people doing good work, Lawrance L. Evans, Jr. here—Managing Director of the U.S. Government Accountability Office’s Applied Research and Methods team. My team houses the agency’s technical and methodological specialists with expertise ranging from cost-benefit analysis and data analytics to survey methods and future-oriented analyses. As GAO ingests important questions from Congress, our network of economists, methodologists, statisticians, data analysts, program evaluation specialists, library scientists, and actuaries help ensure we return accurate, objective, and actionable information—grounded in the appropriate methodologies—to decision-makers and the American public.
Rigorous qualitative and quantitative methods have grown in importance with the focus on quality evidence and the results-oriented culture that is spreading across government. With these tools at its disposal, the government is better able to operate public programs efficiently and effectively, reduce fraud and waste, strengthen accountability mechanisms, and give taxpayers the best return on their investment. Many of our reports and testimonies to Congress highlight the use of high-quality information and rigorous methods to address complex questions, including those related to wicked problems like: climate change and ecosystem management, financial crises, poverty and inequality, or terrorism. As we celebrate our Centennial, we invite you to share in our broad portfolio rooted in a century of non-partisan, fact-based work.
Rad Resources:
Consistent with AEA’s mission to improve evaluation practices and methods, increase evaluation use, and promote evaluation as a profession, I offer a few examples where GAO used rigorous methods to answer difficult causal questions. These reflect collaborative efforts led by our Financial Markets and Community Investment team:
- Homelessness (GAO-20-433): We conducted an econometric analysis to understand select factors associated with homelessness.
- Risk of Regulatory Capture at Federal Reserve, OCC, and FDIC (GAO-18-118, GAO-19-69, GAO-20-519): This body work took a complicated concept—regulators acting in the interest of the regulated industry rather than in the public interest—and created a framework to evaluate the control environment at several agencies.
- Evaluation of BSA/AML Regulations (GAO-18-263): We used surveys, government data, and sophisticated modeling to provide one of the first empirical assessments of the unintended consequences of US anti-money laundering requirements.
- Cumulative Impact of Regulations since 2010 (GAO-18-312): This work took on the Herculean task of assessing the impact of changes in the regulatory environment on community bank outcomes, including mergers and new institution formation, small business lending, and financial performance.
- Too Big to Fail (GAO-14-621): In response to concerns that expectations of government bailouts of large complex financial institutions were distorting investor incentives to price risk appropriately, we estimated funding cost differences over the 2006-2013 period using 42 different models to assess whether large banks benefited from advantages in bond markets.
- Capital Requirements (GAO-12-237): This work leveraged mathematical modelling and time series techniques, to prospectively assess the likely impact of impending changes in capital requirements on lending activity.
Hot Tips:
- Identify with the question/problem, not the tools. Use techniques and methods because they are required to answer the question as validly as possible, not because they are innovative and exciting. We need to uncover the methodology that allows us answer the policy-relevant questions we face—as opposed to searching for specific problems that allow us to use preferred methods.
- Show the requisite humility. Be transparent about assumptions, limitations, and the results of sensitivity analyses. Trust that readers are not looking for perfection, and will recognize valuable insights when they see them.
- Embrace the multidisciplinary nature of most questions. The approach will be made more contextually sophisticated if we expand beyond our sorority/fraternity of expertise. This is critical since many of the problems we seek to solve benefit from collaborative approaches that allow us to explore interdependent structures and dynamic complexity.
The American Evaluation Association is hosting GAO Week in celebration of the US Government Accountability Office’s 100th anniversary. The contributions all this week to aea365 come from authors who address GAO’s efforts to solve those complex “wicked” socio-cultural problems that defy permanent solutions but demand our best efforts to solve them. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.