Hello, I’m William Moore and I am the Director of Research and Measurement at the Institute for Research and Reform in Education (www.irre.org). IRRE partners with schools, districts, and states to transform public schools into more engaging, rigorous, and caring places for students to learn and teachers to teach. My colleagues and I have spent the last decade focused on creating new measurement systems for districts and schools. We have created new approaches to making data actionable for educators, and have provided intensive supports for our clients so that they can better use and take action based on data.
As I write this there is great interest (and federal funding available!) to help our schools become more data-driven. My experience is that most schools and youth-serving organizations do not have a history of systematically using data to inform and improve practice. We have learned that it takes a concerted and sustained effort to create the conditions to support a culture of data use in organizations and intensive support for practitioners to become effective users of data.
Two major challenges educators face in doing this work are: how can accurate and timely data be made available to all who need it when they need it; and how then to use these data in ways that will directly benefit students. Today, I am going to share some of our lessons learned and resources to help you help your client’s establish a culture of data use.
Hot Tip: We have found in our field work with schools that four conditions must be in place for an organization to have a good chance at creating a culture of data use. Focusing your technical assistance toward these four conditions will be a great place to start. We call these conditions ‘building blocks.’ The four building blocks are:
- System leadership — the capacity of system leaders to plan, implement and strengthen data use strategies. This includes a) setting and articulating clear expectations for data use; b) personifying commitment to the use of data; c) providing timely and effective supports to others in the system around data access and use; d) monitoring and reporting progress; e) recognizing others’ accomplishments in using data and intervening to remove barriers to effective data use.
- Data Governance — The intentional actions of an organization to ensure the accuracy and timeliness of its data by creating policies, procedures and practice guidelines that reduce the chance for data-related errors and increase the likelihood of data use. Data is an organizational resource and it must be carefully managed, audited, and controlled – youth serving organizations make very important decisions about children based often on poorly governed data resources.
- Enhanced data and technology systems – Using data in new ways requires new ways of managing and organizing data. Organizations are beginning to implement data warehouses and designing new reports of youth outcomes that provide actionable information for youth workers and teachers. One organization that has partnered with many schools to create this enhanced technology for data use is Mizuni, Inc. Here is an example of one report designed by IRRE that is used by teachers, advocates, parents and students. http://bit.ly/examplereport
- Skill Building and Structures – For professionals unaccustomed to using data to guide and direct their practice decisions an intensive training sequence and supports are needed. Training in data-driven dialogue that takes professionals from learning about what data are available for their use to accessing that data and then using a process or protocol to dialogue with their colleagues around the data and finally to making decisions based on the data in a collaborative, informed fashion is necessary and requires practice, practice and more practice.
Even if an organization has supportive leadership; accurate, credible, and timely data; a data system that generates actionable reports based on this data for all who need it; and staff who have been trained to access and use these reports, the organization will need to examine how it can use time and space to be sure the opportunities to engage in dialogue exist. For example, if teachers do not have common planning time around a shared set of students this reduces the impact that the best data systems and training will have. Organizations who expect staff to be data-driven must create frequent time and space in the workday for this to occur.
Helping your clients create the foundation for effective data use pays off when external evaluators look at their progress. As evidence comes more and more to drive program investments, organizations whose cultures and individual staff value data and use it well will be more likely to work well with evaluators, to have better information about their work and to do their work better.
Rad Resources: To learn more about providing supports to schools around effective data use: http://www.irre.org/publications/doing-what-matters
To learn more about data governance take a look at the work of the Data Quality Campaign: http://bit.ly/dataqualitycampaign
To learn more about Mizuni, Inc., and their data warehousing and reporting solutions for schools and other youth serving organizations: http://www.mizuni.com
This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.