Have you ever wanted to evaluate a large educational survey dataset but were stuck on which statistical software would be the best to use? Then this may be the blog post for you! My name is Kinsey Simone, and I am an instructor of Quantitative Research Methods and Educational Assessment at Tennessee Tech University. I am set to earn my PhD in Program Planning & Evaluation in August 2023 and have spent the past several weeks deep in analysis of a large educational dataset. I learned a few advantages and disadvantages for using some of the most popular software packages, including IBM SPSS, R Software, and AM software, which I am sharing with you.
As evaluators, we have access to many large educational survey datasets; the National Center for Education Statistics provides public access to many of these datasets: https://nces.ed.gov/. When we use a large survey dataset, we must consider unequal sample selection probabilities, nonresponses, and coverage errors. If we do not apply sampling weights or consider the complex designs of these large datasets, then we risk computing biased estimates. Therefore, it is important to choose a statistical software that meets our needs. I list the pros and cons of these different software packages below and share what I found to be best for analyzing the NCES ELS 2002 dataset.
IBM SPSS: IBM SPSS does not have the capacity to correctly analyze complex sample datasets. However, you can buy the IBM SPSS Complex samples survey package, which provides specialized statistics and allows you to consider standard errors in computing statistics. This is very pricey, however, and therefore might not be the best option. View more about this package here: https://www.ibm.com/products/spss-complex-samples
AM Software: AM Software was developed by the American Institutes for Research (AIR) and the U.S. Department of Education (USDE) and was designed for analyzing complex samples and large datasets. You can manually download it here: https://am.air.org/. This software allows you to manually input weights, PSUs, and stratum for analyzing large-scale datasets. However, it is no longer supported by the AIR or the USDE.
R Software: R Software is free and great for statistical computing. It has two packages for analyzing large, complex datasets, including EdSurvey and Survey.
EdSurvey was created for NCES datasets and allows you to download any dataset directly from NCES. While it considers weighted and unweighted correlations to analyze NCES data efficiently, data recoding or using other packages is not easily done.
Survey package in R is good for large survey samples with complex designs. You can create a design where you manually input the stratum, PSU, and variable weights. You can use any other package alongside the Survey package as well, and it is my favorite choice for large survey datasets.
Lesson Learned
IBM SPSS, AM Software, and R all offer packages and other ways to analyze complex survey datasets. I found that the Survey package in R is the most accessible package to use for analyzing complex survey packages and getting reliable results.
The American Evaluation Association is hosting PreK-12 Ed Eval TIG Week with our colleagues in the PreK-12 Educational Evaluation Topical Interest Group. The contributions all this week to AEA365 come from our PreK-12 Ed Eval TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.