Hi! We’re Tameka Porter, Kim Good, and Susan Shebby of the Region 11 Comprehensive Center at McREL International. The Region 11 Comprehensive Center serves Nebraska, North Dakota, South Dakota, and Wyoming as they implement evidence-based interventions that support improved educator and student outcomes.
In our partnership with the Nebraska Department of Education, we created a Professional Learning Community that has been exploring ways to use data to make equity-focused decisions that promote positive student outcomes. As evaluators who work with data, we can sometimes be dismayed with how it’s used, discussed, collected, and analyzed. Sometimes data can be hard to understand because it’s layered in terms and jargon. We’re left scratching our heads wondering “What just happened?” “What am I even looking for here?” So, generously borrowing from one of our guest speakers, Heather Krause of We All Count and Datassist, we offer three strategies evaluators can use to help educators use a data-equity lens.
Hop Tip #1:
Demystify how you evaluate data. First, we can demystify data. Jargon can be difficult to navigate, making the steps of the data process, by which we mean understanding, collecting, organizing, analyzing, and interpreting data, tenuous at best. Those of us who live in data become immune to the technical terms, but if we can explain data terms and concepts in accessible and plainspoken terms, we can bridge knowledge gaps and start using data to support equitable student learning outcomes.
Rad Resources:
We All Count’s Data Speak Decoder is a great resource for being mindful about the words we use when talking about data. University of Southern California’s Glossary of Research Terms can be a launching point for decoding jargon for your evaluation.
Hop Tip #2:
Democratize your evaluation by making data accessibility a priority. Reflecting on one of our Learning Community sessions, we engaged in discussions about who has access to data. We joked that sometimes it exists on high, or that one person has access to it, but if they aren’t in the office, it may be inaccessible. How can you do a meaningful evaluation without data? We can start by democratizing our data. Democratizing data through making our data tools, materials, and resources widely available to those who want and need to access it can make your work technical yet accessible.
Rad Resource:
Teachers and leaders can use WestEd’s Data-Driven Decision Making Toolkit to take inventory of the data available at their schools.
Hop Tip #3:
Demonstrate your data evaluation knowledge by welcoming multiple perspectives in the process. As evaluators, we want to be on the lookout for any assumptions, interpretations, or conceptual biases we may be introducing into the evaluation process once we receive our data (ideally from a trusted and high-quality source). By demonstrating or sharing what we’ve learned in the data evaluation process, we can step outside of ourselves to think about how our experiences, worldviews, and opinions may shape our evaluations. We can support teachers in our evaluation work by welcoming other perspectives and worldviews other than our own. This can prevent us from making one-sided interpretations of the work that centers ourselves instead of focusing on teacher and student experiences.
Rad Resources:
IES’ Practitioner Data Use in Schools Workshop, Massachusetts’ District Data Team Toolkit, Five steps for structuring data-informed conversations and action in education, and eight steps to becoming data wise are useful toolkits and resources for demonstrating your knowledge as an informed data consumer and evaluator.
The American Evaluation Association is celebrating Ed Eval TIG Week with our colleagues in the PreK-12 Educational Evaluation Topical Interest Group. The contributions all this week to aea365 come from our Ed Eval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.
As a student taking a course on evaluation, and a new school administrator that would like to encourage the use of data in order to improve student learning, I really appreciate the three tips shared in this blog posting. One of the most challenging aspects of the course I am taking is the amount of jargon used in evaluation circles. This is something I can be mindful of as an administrator, and also apply to the evaluation I am developing. Your point about access to data also resonates with me. I realize that I have access to a lot more data than most teachers at my school. This is due to the way that data is distributed in our organization, but also due to differences in the way our work day is organized. By sharing data with all intended users, and providing more time for actually digesting the data, I can make our school data much more accessible. The third tip mentioned in your blog is that multiple perspectives strengthen interpretation of data. Because this is true, demystifying data, and increasing access become all the more important. I appreciate your straight-forward and practical approach to data usage in education. Better Education (2021) has several suggestions that can be used to increase access to evaluation data and recommendations. They also suggest the use of plain language, as well as advising the removal of visuals that distract from key information, and using techniques such as visuals, descriptive chart titles, and headings to emphasize key takeaways.
References
Better Evaluation (n. d.). Ensure accessibility. Better Evaluation. https://www.betterevaluation.org/en/rainbow_framework/report_support_use/ensure_accessibility
Hi Tameka, Kim, and Susan,
I really enjoyed your blog post. As a Professional Masters of Education student at Queen’s University and full-time elementary teacher, I am looking to make connections between program evolution theory and my daily teaching practice. This article was meaningful to me, as it provided valuable insight into using data to make evidence-based decisions in my classrooms.
Throughout my degree at Queen’s University, I have come to understand the importance of data in all aspects of learning. In the realm of teaching and learning, data acts as a guiding force providing the direction, focus, and fuel needed to improve student outcomes. Data without a doubt is the fuel for teaching and learning. Without it, educators would not be able to monitor their plan and make the required adjustments to elicit meaningful change or increase student outcomes. Unfortunately, using data to elicit change is a complex skill that many educators struggle with. Therefore, demystifying data, as you put it, is the first step of using data to make meaningful change. Breaking down the jargon and technical terms into “plainspoken” terms will remove the barriers around data analysis and make the information more accessible for every stakeholder, including educators. As you mentioned, this can be done through collaboration with others.
Not only does collaboration provide an opportunity for educators to unpack the meaning of the data and technical terms, but it also allows for multiple perspectives to be considered, which can help to create change. As educators, we often bring our own biases into data analysis, resulting in the maintenance of the status quo. Sometimes we become too overwhelmed by the data, and what it means, opting to continue teaching the way we are used to, as opposed to changing our thoughts and actions. However, if we want true improvement to occur, we need to engage with others and bring in multiple perspectives to overcome our personal biases. Collaborating with others will help us evaluate our own thoughts and assumptions, while also providing us with different lenses to look at a problem. Overcoming these obstacles in data analysis is the key solution to improving teaching and learning.
Thank you for sharing your insights and tips on how to use data appropriately to make informed instructional decisions, as well as providing a space for evaluators and educators to collaborate.
Michelle
Hi Tameka, Kim, and Susan!
As an elementary school teacher currently taking a graduate-level course in program evaluation, I found your post particularly poignant.
I’ve found in teaching that while educators may know how to collect data, analyzing it and utilizing it to create steps for teacher or student improvement can feel intimidating, particularly for those unfamiliar with the process.
When teachers are given reports from outside agencies to review or discuss to facilitate change in our programming or mindsets, the jargon or wordiness can create barriers or make the information inaccessible, as you mention here in your post. PLCs can be an excellent opportunity for educators to collaborate and create meaning from data gathered, but we have to ensure buy-in; this could come from, as you say, “demystifying” the data. Reports and information shared with educators have to be user-friendly, as we are not generally trained in research methods and data collection.
Sometimes when we have data unpacking sessions, like results from large-scale assessments, we are told which information to focus on and which information we need not worry about. Our higher-ups may think this is helping us target our goal-creation or concentrate on the essential information. Still, it often comes across as belittling, which is why I appreciate your focus here on collaboration and partnerships.
Thank you for sharing your thoughts in this post and encouraging collaboration and open conversations between evaluators and educators.
All the best,
Erin