I am Julie Slay, Senior Director of Evaluation at Arabella Advisors, a consulting firm that helps philanthropists and investors achieve the greatest good with their resources. Often, we are asked to evaluate a funder’s portfolio or initiative using indicators based on secondary data sets that reflect community-level and individual-level outcomes. Administrative data have potential to help us do that for behavioral health – or does it?
In the world of health care, using administrative claims data is attractive for a number of reasons. First, what are claims data? They are large data sets, held by public and private payors, that contain information about what a health care provider submitted to get paid for their service. Recent claims data, thanks to HIPAA, are in digital form – downloadable data spreadsheets that provide information about each client health encounter. But how accurate are those claims data? Do they really tell you the story of behavioral health care? Probably not.
Lesson Learned: Claims data have a number of constraints that are important to remember.
- Claims data reflect what the provider and payor need, not always what happened. Studies continue to compare claims data with service encounter data and find significant discrepancies. Often, claims data underestimate the amount and type of services provided, because providers know their payors’ rules and will not submit claims for a non-reimbursable services.
- Data required for reimbursement may not provide you with an accurate picture of quality. Mental health conditions are chronic; over one’s lifetime, there may be shorter, intense periods of using services, and longer periods where fewer services are used. However, we cannot know just from patterns of service use whether the client is receiving high quality services. High quality services might be reflected in a greater frequency of seeing a provider, but it also may indicate poor quality that requires multiple visits to get the care that one needs.
- Claims data are often dated. Once a service is provided, the data must be cleaned before it is claimed. Once sent to a payor, it might go back and forth between the payor and provider due to an error, or a denial then an appeal, and after several months, the service is paid for, or not. So in a world where people want real-time data, administrative claims have NO potential for that.
Lesson Learned: Try using clinical data from an electronic health record (EHR) instead of claims data to assess outcomes. Rules related to reimbursement should not prevent providers from entering these service, and data are close to real-time. If you use claims data, do not rely on it as the only measure of an outcome; use other indicators from different data sources to augment your findings.
The American Evaluation Association is celebrating Alcohol Drug Abuse and Mental Health (ADAMH) TIG Week. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.