Marion O’Reilly-Stanhauer on NWEA’s Data Gallery and Standardized Educational Testing

My name is Marion O’Reilly-Stanhauer. I am an independent evaluation consultant focusing on out-of-school time (OST) programs such as after-school enrichment, tutoring, and summer success. The programs I work with are impacted by standardized testing in their states, often due to calls for additional OST support when standardized tests indicate low proficiency levels or the converse when standardized testing scores raise concerns that prompt advocacy for redistribution of funds away from OST to core competencies. As such, I find standardized testing to be increasingly and unexpectedly important to the context in which I practice.

Rad Resource – The Kingsbury Center at Northwest Evaluation Association (NWEA) Data Gallery: The Data Gallery is a project from the Kingsbury Center that explains and shares their data focusing on United States standardized testing educational achievement. The Data Center is eye opening on many levels. There are currently two online ‘exhibits.’ Each exhibit includes a video from a lead project researcher, four interactive graphs that provide comparative data, and links to research and commentary to help you to learn more.

Exhibit 1 – State of Proficiency: This exhibit focuses on the inconsistency of state standards across states and grades, so that states with easier standards report higher rates of student mastery.

Exhibit 2 – Achievement Gaps: This exhibit builds on the state of proficiency exhibit, providing profiles showing that a standout student in one state could be sub-par in another. As an example, 20% of the students in Clarkson Elementary (a hypothetical example provided by NWEA) would be deemed proficient in fifth grade mathematics in Massachusetts, while 80% would be deemed proficient in Colorado.

Lessons Learned for Accessibility and Transparency: I believe that the Data Center helps parents, teachers, and the public to ask fundamental questions about standardized testing. It allows stakeholders to examine their state in relation to others, and ideally prompts questions to policymakers about group and test disparities. I look at what they have developed as an opportunity to consider how I could work with my own clients, mostly nonprofits, to share data with their stakeholders and prompt similar actions.

Lessons Learned for Data Exhibiting: The Data Gallery uses Tableau software to provide the interactive visualizations. I found that they were useful, and that the available manipulations helped me to answer some of my own questions (how did my state compare? Are there inconsistencies based on school type), but not others (how does race and ethnicity enter into the equation, do the state inconsistencies reflect curriculum foci or contextual differences?). In some cases, it was difficult to understand exactly what was being compared. As an evaluator, I wanted more ready access to the source of the data and a more full explanation of the data source and calculations right on the graphs (rather than accessible only within a pdf of the full report).

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.