AEA365 | A Tip-a-Day by and for Evaluators

TAG | families

I’m Sondra Stegenga, an occupational therapist, home visitor, educational administrator, and Ph.D. student at the University of Oregon.  Evidence has shown that meaningful family involvement is key to long-term outcomes for children. In early intervention and early childhood (EC) systems we are charged with basing services, supports, and goals on family needs and priorities. Given the varied learning needs and contextual and cultural values of families, and the lack of research on involving families in data practices, this process may be unintentionally overlooked or underutilized. In a recent study, Brawley and Stormont found that although 82% of EC teachers identified sharing data with families as important, only 42% reported regularly doing so. Data collection in EC programs can become a rote task, completed without much meaning or family involvement. Failing to include families in data processes not only violates foundational tenets of early intervention and early childhood but more importantly deprives families of valuable learning and reflection, greater involvement in their child’s plan, and improved chances of successful outcomes.

Lessons Learned:

  • In 20+ years of working with children and families I learned the impact of involving families in data practices. This lines up with what researchers and evaluators have noted that involving families in data processes leads to increased communication and better outcomes.

Hot Tips:

  • To engage parents in data practices we must first engage families in the whole educational process. Consider cultural, contextual, and family needs. Engagement may look different to each family, but should be conveyed thorough mission, goals, and formal practices explicitly outlining the importance of and practices supporting family involvement. Gathering input from through a variety of methods (via smartphone, in-person, and times convenient for the family) is imperative to meaningful family engagement.
  • Involve families from the beginning as “partners” in data collection, reflection, and use. This will demystify the process and support full, meaningful family engagement. Explain reasoning for data, timelines, and gathering data. Take time to understand parents’ prior experience, fears, and questions related to data. Ask parents what is meaningful to them and discuss how they would like to measure their child’s progress.
  • Use various modes of data presentation. Graphs and visualizations are shown to be powerful communicators of data. In addition, telling the story of the data and linking to family’s needs, priorities, and contexts is key to understanding.

Rad Resources:

The American Evaluation Association is hosting the Disabilities and Underrepresented Populations TIG (DUP) Week. The contributions all week are focused on engaging DUP in your evaluation efforts. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

I’m Jessica Sickler, and I work as a research and evaluation consultant, as a senior researcher and manager of the Lifelong Learning Group.  My career began in a zoo; in answer to the obligatory cocktail party question, “No, I did not shovel the poop.”  I started as an internal evaluator at the Wildlife Conservation Society (Bronx Zoo), and have continued to work with zoos and aquariums nationwide to understand what, how, and why visitors learn in this context.

Lessons Learned:

  • Zoos care deeply about visitor learning, and seek to achieve learning outcomes in “the wild” of the zoo visitation context.  Free-choice learning settings cede a lot of control to the learner.  At a zoo, you cede a lot of control to nature – weather, seasonality, and the unpredictability of a living collection.  We examine learning experiences of audiences spending a full day of leisure time dealing with weather conditions, sugar highs, gift shop temptations, and child meltdowns. The work of collecting data requires a positive attitude, a vat of sunscreen, and adaptability– I once completed an interview with two adults while their child tied his six-foot-long stuffed snake around my legs.
  • Zoos advance conservation of wildlife and habitats. For those where entrapment by five-year-olds is not a job hazard, a transferable lesson from zoo evaluation has to do with alignment.  A common feature across zoos is a mission to advance conservation of wildlife and habitats.  Zoo planning becomes an exercise in aligning exhibits, programs, and in-park experience to mission.  It begs the question “what are reasonable conservation-supporting outcome expectations for a single, four-hour zoo visit?”  As an evaluator, I have found value in helping map the connections.
  • Zoos promote learning. Another question of alignment surrounds expectations and experience.  While the zoo has its educational agenda, visitors arrive with different sets expectations.  Although learning may be a factor, other factors can’t be ignored – especially social value, since most visitors come as families. In one study, we gave visitors the prompt: “Today’s visit reminded me of something important, and that was…”  Although project planners hoped for responses exhibiting thinking about one’s place in a global environment, the answer was essentially, “That it’s so important to spend time together as a family.”  Visitors’ expectations and values shape meaning-making during their visit; the evaluator’s role is helping teams understand these attributes and see them as opportunities to extend caring and concern to the natural world.

Rad Resources

The American Evaluation Association is celebrating Arts, Culture, and Audiences (ACA) TIG Week. The contributions all week come from ACA members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluator.

· ·

Greetings, I am Jane Eppard, Vice-Chair of the Family Hope Foundation. We founded FHF in 2009 with a mission to invest in children with special needs through family support, engaging community experts, advocacy through collaboration, and financial assistance to provide access to therapies. In a very short amount of time we have gained community support and confidence and recently won Wood TV 8’s coveted Connecting with Community award. One reason for this rapid growth is due to our investment in evaluation from the very beginning.

Hot Tips:

  • When in doubt, evaluate everything. When we first began our organization, we were not sure what we should evaluate. We decided to evaluate everything from process to programming. We evaluate processes and programs formatively and adjust where needed. We also evaluate summatively to determine the overall impact and the merit and worth of our programs.
  • When in doubt, evaluation everyone. Often, organizations only evaluate consumer satisfaction. We decided to be effective, we needed to evaluate everyone. We have board members evaluate the strategic planning process, family members evaluate our scholarship application process, we interview community partners, and gather stories of the children we serve.

Lessons Learned:

  • When you don’t have the expertise yourself, solicit board members. FHF started with a handful of passionate people determined to make an impact. We learned quickly that we needed more expertise in several areas. We were able to garner this expertise by soliciting board members with a diverse set of skills. The FHF board includes professionals with expertise in law, accounting, non-profit leadership, education, and yes, evaluation.
  • Share the data, share the need. To justify funding needs, we shared census, educational, and even anecdotal data through our initial solicitations, website, newsletters, and annual reports. We also collected data on our scholarship applications and were able to make appeals based on the data from the applications. Often this type of data is not publicly accessible, for instance, the number of therapies children need that are not covered by insurance.

  • Share the data, celebrate success! Since we gathered data on everything, we were able to show gains from our very first funding cycle. This quickly gave confidence to our donors and community members that we are indeed making an impact on children with special needs and their families.

The Family Hope Foundation increase in scholarships

  • Qualitative data garners support. Donors love success stories. For our consumers, those successes are often not statistically significant or able to be measured quantitatively. I encourage you to gather evidence in the form of interviews, stories, photographs, and open-ended survey questions. The share qualitative results are often the reason we gain new supporters.

Resources:

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

Archives

To top