AEA365 | A Tip-a-Day by and for Evaluators

TAG | data-driven decision making

I’m Sondra Stegenga, an occupational therapist, home visitor, educational administrator, and Ph.D. student at the University of Oregon.  Evidence has shown that meaningful family involvement is key to long-term outcomes for children. In early intervention and early childhood (EC) systems we are charged with basing services, supports, and goals on family needs and priorities. Given the varied learning needs and contextual and cultural values of families, and the lack of research on involving families in data practices, this process may be unintentionally overlooked or underutilized. In a recent study, Brawley and Stormont found that although 82% of EC teachers identified sharing data with families as important, only 42% reported regularly doing so. Data collection in EC programs can become a rote task, completed without much meaning or family involvement. Failing to include families in data processes not only violates foundational tenets of early intervention and early childhood but more importantly deprives families of valuable learning and reflection, greater involvement in their child’s plan, and improved chances of successful outcomes.

Lessons Learned:

  • In 20+ years of working with children and families I learned the impact of involving families in data practices. This lines up with what researchers and evaluators have noted that involving families in data processes leads to increased communication and better outcomes.

Hot Tips:

  • To engage parents in data practices we must first engage families in the whole educational process. Consider cultural, contextual, and family needs. Engagement may look different to each family, but should be conveyed thorough mission, goals, and formal practices explicitly outlining the importance of and practices supporting family involvement. Gathering input from through a variety of methods (via smartphone, in-person, and times convenient for the family) is imperative to meaningful family engagement.
  • Involve families from the beginning as “partners” in data collection, reflection, and use. This will demystify the process and support full, meaningful family engagement. Explain reasoning for data, timelines, and gathering data. Take time to understand parents’ prior experience, fears, and questions related to data. Ask parents what is meaningful to them and discuss how they would like to measure their child’s progress.
  • Use various modes of data presentation. Graphs and visualizations are shown to be powerful communicators of data. In addition, telling the story of the data and linking to family’s needs, priorities, and contexts is key to understanding.

Rad Resources:

The American Evaluation Association is hosting the Disabilities and Underrepresented Populations TIG (DUP) Week. The contributions all week are focused on engaging DUP in your evaluation efforts. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

Hi my name is Kim Sabo Flores, I am the Co-Founder of Algorhythm. Over the last 20+ years, I have been working as an evaluator in the field of youth development. Recently I’ve observed an unfortunate trend in the field: A LOT OF TALK about “evidence-based,” “research-based” and “data-driven” decision-making, and very LITTLE ACTION. This is particularly true for youth practitioners, working on the front lines of social change, where data could have the greatest impact. Why, in this rich information and technology era is this still a challenge?

Here are a few Hot TIPS:

Bring the power of data to the front lines of social change: Data is power! And for the most part that power is held by senior-level staff and has been used to leverage resources rather than to drive programmatic decision-making. It is rarely the case that evaluation findings are shared and analyzed with front-line staff, and there is a radical misunderstanding of their ability to effectively understand and utilize data.

Hot Tip: Support ALL staff to learn from and make meaning of data; be sure they’re included when you share your findings.

Value rather than evaluate: Research and even evaluation reports are written and consumed by academics and funders. However, they leave practitioners with limited practical information about how to improve outcomes for ALL youth —specifically the most difficult to serve.

Hot Tip: Utilize predictive and prescriptive analytics that focus on what “works” for each and every youth, valuing all the various pathways taken toward success rather than just those taken by the “average” youth.

Measure what matters: Driven by funding demands, program staff spend precious time and resources capturing mandated data such as report cards, test scores, attendance records, etc., with the full knowledge that these metrics do not fully tell their story and are not fully attributable to their programs. Front line workers are tired of gathering meaningless data that doesn’t answer their questions.

Hot Tip: Use research-based social/emotional measures to show proximal gains that contribute to academic achievement, reduction of risk and thriving. These types of outcomes speak directly to the work of youth development and allow front-line staff to see their contribution.

Provide timely insights at a low cost: Take advantage of new technologies that allow programs to gather data, immediately analyze it and put it to use. Such technologies increase data utilization and ultimately increase the impact on youth. Best of all, it drastically decrease the cost and allows more nonprofits to afford evaluation and to afford it more often!

Rad Resources:

Foundation For Young Adult Success: UCChicagoCCSR. Concept Paper for Research and Practice. June 2015.

FREE webinar:“21st Century Impact Measurement for Youth Serving Organizations,” and learn more about a game-changing approach to impact measurement.

The American Evaluation Association is celebrating Youth Focused Evaluation (YFE) TIG Week with our colleagues in the YFE AEA Topical Interest Group. The contributions all this week to aea365 come from our YFE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Archives

To top