Hello, we are Qiong Louie-Gao and Humberto Reynoso-Vallejo, members of the research team evaluating Chapter 224, the Health Care Cost Containment Law, from the Office of the State Auditor in Massachusetts.
In order to implement a comprehensive evaluation on the impact of Chapter 224, our research team is committed to find the most effective statistical procedures to use in order to create a solid baseline that can lead to a sound quantitative analysis of the entire longitudinal project. One of the procedures we found useful is predictive modeling.
Predictive modeling is a process of fitting a statistical model based on existing relationships among variables and making an informed prediction for future behavior(s). In our study, a primary goal is to find the trend or pattern of our variables using a number of data points before the implementation of Chapter 224 (prior to implementation in 2012) as a baseline. We then use the parameter estimates of the baseline to predict the values after 2013. A solid baseline with predicted values will be compared with the actual data once we conclude the longitudinal quantitative analysis.
Hot Tip: This method can be used for both individual data and aggregate data. For individual data, each individual may have repeated measures that may produce a correlation. Generalized Estimating Equations (see Zeger & Liang, 1986) and Linear Mixed-Effect Models (see Cnaan, Laird, & Slasor, 2005) are the two general statistical methods for the longitudinal data analysis. Note that General Estimating Equations represent the effects on the population average, whereas Linear Mixed-Effect Models aims to estimate the effects on a subject-specific basis. For aggregate data, a linear regression model can be used to estimate the slope of the time series.
Hot Tip: Data fluctuation or outliers will compromise the results. To account for this, some statistical techniques such as “moving average” may be used to smooth out the data.
Hot Tip: Carefully select covariates to control confounding and avoid multicollinearity (see Steen, et al, 2002) based on background information. Some variable-selection approaches can be used, e.g. step-wise, forward, and backward selection procedures.
The American Evaluation Association is celebrating Large Scale Evaluation Week. The contributions all this week to aea365 come from evaluators who have worked on the evaluation of the Health Care Cost Containment Law in Massachusetts. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.