AEA365 | A Tip-a-Day by and for Evaluators

TAG | Government Performance and Results Act

Hi, I’m Lisa Melchior, President of The Measurement Group LLC, a consulting firm focused on evaluation of health, behavioral health, and social services for at-risk and vulnerable populations. I’m delighted to share some thoughts about the role of federal performance data and reporting requirements as they pertain to local evaluation. Since the inception of the Government Performance and Results Act (GPRA) in 1993, my colleagues and I have conducted dozens of evaluations that involved complying with these data collection and reporting requirements.

GPRA is U.S. federal law which created a mechanism for government agencies to engage in strategic planning, performance planning, and performance reporting. In 2010, the GPRA Modernization Act was updated, emphasizing priority- setting and cross-organizational collaboration. It also increased its focus on using goals and measures to improve outcomes of federally funded programs.

Many of the programs my colleagues and I evaluate are funded by the US Department of Health and Human Services, Substance Abuse and Mental Health Services Administration (SAMHSA), Center for Substance Abuse Treatment (CSAT), which is required to report to Congress on progress on its designated GPRA measures. SAMHSA/CSAT requires organizations that provide direct client services funded by CSAT discretionary grants to implement the Client Outcome Tool. Although it can vary depending on the grant program, grantees are generally required to collect this information from individuals at program intake, at 6-months post-intake, and at program completion or discharge. The GPRA tool includes questions regarding the participant’s housing situation, physical and mental health, substance use, sexual activity and criminal justice system involvement. While this standardizes data collection for all grantees within a funded initiative (and to a broader extent across all the funding agency’s discretionary grantees), it also has a substantial impact on evaluation design, implementation, and analysis at the local level.

Hot Tip:

GPRA data can be used for program monitoring and evaluation at the local level. Grantees and their evaluators can access summary reports of selected GPRA data (SAMHSA grantees use a system called SPARS). Grantees also have the capability to download their raw data for local use.

Cool Tricks:

Try to integrate required performance measurement data collection protocols with additional measures used for project-specific evaluation. Aim to minimize repetition and redundancy. Pay close attention to the flow of when and how questions are asked of program participants.

Lessons Learned:

While GPRA data is useful for local evaluation, it is often not sufficient for this purpose. For example, self-reported substance use is only measured in the past 30 days. Mental health symptoms are also measured by responses to single-item self-report measures of past 30-day symptoms. If it is important to document lifetime substance use patterns, or to know about actual DSM-based mental health conditions, it is necessary to supplement the GPRA measures.

Rad Resources:

General information: https://en.wikipedia.org/wiki/Government_Performance_and_Results_Act

https://obamawhitehouse.archives.gov/omb/mgmt-gpra/index-gpra

Information about SAMHSA’s GPRA Measurement Tools: https://www.samhsa.gov/grants/gpra-measurement-tools

The American Evaluation Association is celebrating Behavioral Health (BH) TIG Week with our colleagues in Behavioral Health Topical Interest Group. The contributions all this week to aea365 come from our BH TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Greetings! We’re Kathleen Ferreira and Roger Boothroyd, Co-Chairs of AEA’s Behavioral Health Topical Interest Group (TIG). The Behavioral Health TIG is pleased to sponsor this week’s blogs for AEA365. We have chosen to draw upon our members’ experience, expertise, and thoughts as it applies to GPRA, the Government Performance and Results Act. As you will read more throughout the week, the Act was originally enacted in 1993 and updated in 2010 and is designed to improve government performance management. As a result of the Act, members conducting evaluations of grants funded by the Substance Abuse and Mental Health Service Administration are required to report data on the GPRA measures. At the Evaluation 2017 conference, we had a thought-provoking discussion during our Behavioral Health TIG’s Business Meeting on issues such as how federal performance monitoring reporting requirements can be used to enhance local program impact and how we can maximize the use of the large volume of data collected at the local level in making day-to-day decisions in improving local service delivery. We thought that this would be an interesting topic for this week’s blog. We hope you agree.

If we go to our dictionary (does anyone have a real dictionary anymore?) requirements is defined as “a thing that is compulsory; a necessary condition.” As many are aware, GPRA has mandated federal reporting requirements. That doesn’t sound appealing, does it? In particular, it sounds like more work! Yet, as evaluators, this creates a real opportunity—one in which we can work closely with our program staff to determine how data that we must report can best meet our local program staffs’ needs. It may not be readily apparent to us at first, it may take us some time to figure it out, but in the end, if we are thoughtful and persistent, we’ll find the answer. Why? Because we’re evaluators – it’s our job! It’s what we do.

This week, readers will learn:

  • the definition of GPRA and how GPRA data can be used for program monitoring and evaluation at the local level.
  • why it is critical for us to continue to collect and report GPRA data, even during governance challenges and uncertain times.
  • how to create data reporting sheets in order to reduce burden on program staff.
  • (taking this idea of data reporting sheets a step further) how to maximize the use of GPRA data by creating program fact sheets.
  • lessons learned during a 5-year evaluation of a Screening, Brief Intervention, and Referral to Treatment (SBIRT) program in which behavioral health services were integrated into a medical setting.

We hope you enjoy the week!

The American Evaluation Association is celebrating Behavioral Health (BH) TIG Week with our colleagues in Behavioral Health Topical Interest Group. The contributions all this week to aea365 come from our BH TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Archives

To top