DVR TIG Week: Meet Them Where They Are: Adjusting Data Visualization to Resonate with K–12 Evaluation Stakeholders by Rachel L. Schechter

Rachel Schechter
Rachel Schechter

My name is  Rachel Schechter, Director of Research at Lexia Learning Systems LLC, A Rosetta Stone company. At Lexia, our mission is to improve student literacy by leveraging technology to personalize learning and simplify the use of data to drive instruction. The research team at Lexia is committed to evaluating the efficacy and validity of our products, informing product design, partnering with customers to evaluate their implementations and student progress, and disseminating research findings and best practices.

A large part of my job is to develop dashboards and data visualizations to help communicate findings to evaluation stakeholders. In an effort to be more personalized in our reporting, I’ve been thinking a lot about the balance between scalability and customization – relying on templates vs. creating fresh content for each project or constituency.

Recently, one of the largest school districts in the country requested custom reporting for Lexia® RAPIDTM Assessment, my company’s online literacy screener. The request came through internal Lexia staff working with the district. Initially, I was told that they “just need the basic info” organized into networks of schools. I took our existing templates and mocked up what seemed like a small adjustment.

The following week I presented samples to the district leaders, and they said that the graphs and tables didn’t look like the report designs that they were used to. I shifted quickly and turned their attention to characteristics like format, levels of summaries (grade, network, district, school) and graph type (stacked columns, bar) to better understand what resonated with them. Then I asked for samples of their commonly used reporting so I could pull design elements that were familiar.

A few weeks later I presented the updated reporting to the district leader. She commented that she “saw her feedback” in the revisions and how “heard” she felt by our team. Success! She provided a final round of feedback related to color choice and ordering of groups – easy to adjust in time for the final report delivery. The training for all administrators is this week, and I feel confident that they will be able to use the information in the customized reporting to make instructional decisions at the network and school level — the whole point of assessment!

Lesson Learned:

No matter what you’ve heard from your colleagues or others involved in a relationship, always start with a needs assessment to start “from the beginning” with the evaluation end-user.  

Hot Tip:

Get samples from clients/customers of reports created by their internal team!  Using chart types and dashboard setups that stakeholders are familiar with will facilitate their understanding and support usefulness.

Pay attention to details, something as small as a color choice or the order of the items in a stacked bar chart bring meaning to the information. Check assumptions about those qualities along your design journey.

Note: Data in each image do not match, are not from a single district, and are not reflective of the district mentioned in the story.

First mockup of data and sample graph provided by schoolRevised graph and final graph

The American Evaluation Association is celebrating Data Visualization and Reporting (DVR) Week with our colleagues in the DVR Topical Interest Group. The contributions all this week to aea365 come from DVR TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

6 thoughts on “DVR TIG Week: Meet Them Where They Are: Adjusting Data Visualization to Resonate with K–12 Evaluation Stakeholders by Rachel L. Schechter”

  1. Arielle Rodriguez

    It all starts from a need! I can appreciate this post because I have experienced a similar lesson when I began substitute teaching. The required orientation and training only prepared me for what the expectation is. Once I was in the classroom I discovered the real need and I figured out how to help solve that problem after I addressed the need. After I checked back with a regular teacher about my method, her feedback reassured me that I was on track and a very helpful sub to have in class. I am sure the district orientation and training is essential for many of the substitute teachers in the district, however, getting into the needs of the students and teachers proved to be more helpful in my situation.

  2. I found this article interesting because you talk about the impact that the visual representation of data can have on people. This year the Ministry of Education in BC has provided us with new language for reporting purposes. Since I use Freshgrade (an eportfolio program) for my reporting to parents I quickly inputed a new “Assessment tool” into the software so that I could be efficient in how I relate assessment data to my parents. However, I had to choose colours to correspond to each Proficiency level on the reporting scale. I found myself very aware of what different colours might represent to parents, even before they read the assessment. I realized that putting “Emerging” in red might be interpreted as a stop signal and amplify parents concern over why their child is at the lower end of the assessment scale. I was also aware that connecting “Extending” with green may also unnecessarily amplify the assessment being given. This article helped solidify my decision to be cautious about what the visual representation of assessment data is when communicating with parents. 

    Further, a colleague and I were discussing that maybe we should be consistently using the same colours across the grades in order to easier transition that information to another teacher. Before the use of eportfolios we never would have needed to consider this because everything was written down. However, with this shift to eportfolio recording we need to be aware of the visual representation of this information.

  3. I really appreciate how you used communication with the user to help them be able to best understand and interpret the data set, and how they were able to give you feedback about this approach. I think the best use of research is when it can be applied, which you identified in your post. I like how you keep your eyes on the ultimate goal, which is to be able to use the information to make decisions that will help the students.

    Awesome job Rachel!

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.