Hi, I’m Lisa Melchior, President of The Measurement Group LLC, a consulting firm focused on evaluation of health, behavioral health, and social services for at-risk and vulnerable populations. I’m delighted to share some thoughts about the role of federal performance data and reporting requirements as they pertain to local evaluation. Since the inception of the Government Performance and Results Act (GPRA) in 1993, my colleagues and I have conducted dozens of evaluations that involved complying with these data collection and reporting requirements.
GPRA is U.S. federal law which created a mechanism for government agencies to engage in strategic planning, performance planning, and performance reporting. In 2010, the GPRA Modernization Act was updated, emphasizing priority- setting and cross-organizational collaboration. It also increased its focus on using goals and measures to improve outcomes of federally funded programs.
Many of the programs my colleagues and I evaluate are funded by the US Department of Health and Human Services, Substance Abuse and Mental Health Services Administration (SAMHSA), Center for Substance Abuse Treatment (CSAT), which is required to report to Congress on progress on its designated GPRA measures. SAMHSA/CSAT requires organizations that provide direct client services funded by CSAT discretionary grants to implement the Client Outcome Tool. Although it can vary depending on the grant program, grantees are generally required to collect this information from individuals at program intake, at 6-months post-intake, and at program completion or discharge. The GPRA tool includes questions regarding the participant’s housing situation, physical and mental health, substance use, sexual activity and criminal justice system involvement. While this standardizes data collection for all grantees within a funded initiative (and to a broader extent across all the funding agency’s discretionary grantees), it also has a substantial impact on evaluation design, implementation, and analysis at the local level.
GPRA data can be used for program monitoring and evaluation at the local level. Grantees and their evaluators can access summary reports of selected GPRA data (SAMHSA grantees use a system called SPARS). Grantees also have the capability to download their raw data for local use.
Try to integrate required performance measurement data collection protocols with additional measures used for project-specific evaluation. Aim to minimize repetition and redundancy. Pay close attention to the flow of when and how questions are asked of program participants.
While GPRA data is useful for local evaluation, it is often not sufficient for this purpose. For example, self-reported substance use is only measured in the past 30 days. Mental health symptoms are also measured by responses to single-item self-report measures of past 30-day symptoms. If it is important to document lifetime substance use patterns, or to know about actual DSM-based mental health conditions, it is necessary to supplement the GPRA measures.
General information: https://en.wikipedia.org/wiki/Government_Performance_and_Results_Act
Information about SAMHSA’s GPRA Measurement Tools: https://www.samhsa.gov/grants/gpra-measurement-tools
The American Evaluation Association is celebrating Behavioral Health (BH) TIG Week with our colleagues in Behavioral Health Topical Interest Group. The contributions all this week to aea365 come from our BH TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to email@example.com. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.