AEA365 | A Tip-a-Day by and for Evaluators

TAG | Inclusive

I’m Sondra Stegenga, an occupational therapist, home visitor, educational administrator, and Ph.D. student at the University of Oregon.  Evidence has shown that meaningful family involvement is key to long-term outcomes for children. In early intervention and early childhood (EC) systems we are charged with basing services, supports, and goals on family needs and priorities. Given the varied learning needs and contextual and cultural values of families, and the lack of research on involving families in data practices, this process may be unintentionally overlooked or underutilized. In a recent study, Brawley and Stormont found that although 82% of EC teachers identified sharing data with families as important, only 42% reported regularly doing so. Data collection in EC programs can become a rote task, completed without much meaning or family involvement. Failing to include families in data processes not only violates foundational tenets of early intervention and early childhood but more importantly deprives families of valuable learning and reflection, greater involvement in their child’s plan, and improved chances of successful outcomes.

Lessons Learned:

  • In 20+ years of working with children and families I learned the impact of involving families in data practices. This lines up with what researchers and evaluators have noted that involving families in data processes leads to increased communication and better outcomes.

Hot Tips:

  • To engage parents in data practices we must first engage families in the whole educational process. Consider cultural, contextual, and family needs. Engagement may look different to each family, but should be conveyed thorough mission, goals, and formal practices explicitly outlining the importance of and practices supporting family involvement. Gathering input from through a variety of methods (via smartphone, in-person, and times convenient for the family) is imperative to meaningful family engagement.
  • Involve families from the beginning as “partners” in data collection, reflection, and use. This will demystify the process and support full, meaningful family engagement. Explain reasoning for data, timelines, and gathering data. Take time to understand parents’ prior experience, fears, and questions related to data. Ask parents what is meaningful to them and discuss how they would like to measure their child’s progress.
  • Use various modes of data presentation. Graphs and visualizations are shown to be powerful communicators of data. In addition, telling the story of the data and linking to family’s needs, priorities, and contexts is key to understanding.

Rad Resources:

The American Evaluation Association is hosting the Disabilities and Underrepresented Populations TIG (DUP) Week. The contributions all week are focused on engaging DUP in your evaluation efforts. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

Greetings! Caitlyn A. Bukaty here sharing some exciting insights with you during Disabilities and Underrepresented Populations Topical Interest Group week.

Today I want to offer a few ideas my fellow evaluators might find helpful in making their evaluations more accessible to a wide range of stakeholders. This information comes from my experience collecting feedback from young adults with intellectual disabilities who participated in a workplace problem-solving intervention, but one of my favorite features of these techniques is how helpful they are to a wide range of stakeholders! This concept is known as Universal Design, and the premise is that an option you might offer to one groups of stakeholders, for example those who have difficulty reading, actually makes accessing your evaluation materials easier for other groups, such as stakeholders for whom English is not their first language, or those with visual impairments.

Without further ado, let’s explore some ideas to help your evaluations reach for the stars in terms of accessibility!

Hot Tips:

  •  Add pictures – A well-connected photo can help stakeholders link a question to a certain event, or clarify a response.

In this example, a series of question are linked to a certain part of the intervention using a picture of the person with whom participants interacted:

Example 1

Here responses are clarified with “thumbs up” and “thumbs down” symbols

Example 2 Thumbs up

 

 

  • Go digital – Offering traditionally “print” materials in digital format opens up a universe of accessibility for stakeholders. Users can access screen reader software, text-to-speech features, and even translation applications to better understand the material. This is even more effective if materials are offered on a mobile friendly platform, mobile web access is widely reported as overtaking desktop computer use.
  • Be all ears – Prepare to accept responses from your stakeholders in a variety of creative ways. Offering stakeholders multiple options for response may mean gathering responses from those who would not have been able to participate via a single mode of response. Written or typed responses to forced choice and open ended questions may be traditional, but what if someone wants to dictate a response…can you make a scribe available in person or via telephone to support his or her participation? How about a participant wishing to record a response? This can be achieved via a voice or video recorder on many mobile devices. Depending on the question a pictorial response, such as indicating time spent on a circle graph, might even encourage respondents to participate.

Rad Resources:

  • Creative Commons Zero (CC0) Imagery – This is the name given to images free from copyrighting. In addition to taking or requesting photos specific to the topic of your evaluation there are resources linking you directly to CCO images such as Pixabay and Unsplash. Web search platforms, such as Google Images also allow you to specify reuse policies during an imae search.
  • Web-based Survey Platforms – These are useful for creating digital surveys or questionaires. Many are mobile friendly, and several platform offer free features or use. Try SurveyMonkey, SurveyGizmo, QuestionPro, or Google Forms.

Lesson Learned:

  • The idea behind today’s post is to maximize stakeholder participation by inviting them to take part in an evaluation in whatever way is most convenient and effective. To learn more about universal design geared towards materials development and response check out the Universal Design for Learning materials offered through CAST.

The American Evaluation Association is hosting the Disabilities and Underrepresented Populations TIG (DUP) Week. The contributions all week are focused on engaging DUP in your evaluation efforts. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · ·

Greetings! We are Della Thomas and Marcia Kolvitz. Della works with local school districts to providing language access services to students who are deaf or hard of hearing. Marcia is an educational consultant who focuses on professional development in the areas of transition planning and postsecondary opportunities for students and youth who are deaf or hard of hearing (D/HH). Both of us have worked with a large-scale initiative to support collaborative activities that engage stakeholders from across the United States to address issues in deaf education. Our participants represented a variety of stakeholder groups, and many of them were D/HH. We’ve considered ways to ensure that our diverse group of participants have the opportunity to participate in these collaborative activities equally. Additionally, as travel funds become scarce and stakeholders’ schedules become busier, we’ve supplemented face-to-face meetings with technology use as a means of building community and supporting team activities.

Lessons Learned:

  • Open captions during presentations benefit everyone. Communication Access Realtime Translation (CART) services are considered an accommodation for participants who are D/HH. However, background noise can make it difficult for other participants to hear the speaker, and some participants may find their attention wandering. The use of CART captions is a good example of Universal Design during a conference.
  • Telephone communication doesn’t always work. Participants who are D/HH may request sign language interpreters during teleconferences to facilitate communication among team members. A simple way to provide this is by using videoconferencing for all participants. Not only does this include the D/HH member, but the non-verbal cues such as facial expressions or body language can provide all participants with additional information.
  • Use a professional for important event (aka evaluation). The standard for a CART provider using a steno keyboard is a minimum of 180 words per minute (wpm) and an accuracy rate of 96%.

Although these lessons learned came as the result of planning large-scale interagency collaborative activities, their value extends beyond individuals with hearing loss. Enhancing large group presentations via CART and small group meetings via videoconferencing will not only provide greater linguistic access for participants, but will send a message of inclusivity for all.

Rad Resources:

RIT Job Board

RIT Job Board

 

 

 

 

 

 

 

  • Want to add captions yourself?  Try MAGPie free software for adding captions and video descriptions to QuickTime, Windows Media, Real and Flash multimedia.
MAGpie2

Media Access Generator – MAGpie2

 

 

 

 

 

 

 

The American Evaluation Association is hosting the Disabilities and Underrepresented Populations TIG (DUP) Week. The contributions all week are focused on engaging DUP in your evaluation efforts. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · · ·

Greetings and welcome from the Disabilities and Underrepresented Populations TIG week.  We are June Gothberg, Chair and Caitlyn Bukaty, Program Chair.  This week we have a strong line up of great resources, tips, and lessons learned for engaging typically underrepresented population in evaluation efforts.

You might have noticed that we changed our name from Disabilities and Other Vulnerable Populations to Disabilities and Underrepresented Populations and may be wondering why.  It came to our attention during 2016 that sever of our members felt our previous name was inappropriate and had the potential to be offensive.  Historically, a little under 50% of our TIGs presentations represent people with disabilities, the rest are a diverse group ranging from migrants to teen parents.  The following Wordle shows the categorical information of presentations our TIGs presentation

Categories represented by the Disabilities and Underrepresented Populations presentations from 1989-2016

TIG members felt that the use of vulnerable in our name set up a negative and in some cases offensive label to the populations we represent.  Thus, after discussion, communications, and coming to consensus we proposed to the AEA board that our name be changed to Disabilities and Underrepresented Populations.

Lessons Learned:

  • Words are important! Labels are even more important!
  • Words can hurt or empower, it’s up to you.
  • Language affects attitudes and attitudes affect actions.

Hot Tips:

  • If we are to be effective evaluators we need to pay attention to the words we use in written and verbal communication.
  • Always put people first, labels last. For example, student with a disability, man with autism, woman with dyslexia.

The nearly yearlong name change process reminded of the lengthy campaign to rid federal policy and documents of the R-word.  If you happened to miss the Spread the Word to End the Word Campaign, there are several great video and other resources at r-word.org.

High School YouTube video

YouTube Video – Spread the Word to End the Word

 

 

 

 

 

 

https://www.youtube.com/watch?v=kTGo_dp_S-k&feature=youtu.be

Bill S. 2781 put into federal law, Rosa’s Law, which takes its name and inspiration for 9-year-old Rosa Marcellino, removes the terms “mental retardation” and “mentally retarded” from federal health, education and labor policy and replaces them with people first language “individual with an intellectual disability” and “intellectual disability.” The signing of Rosa’s Law is a significant milestone in establishing dignity, inclusion and respect for all people with intellectual disabilities.

So, what’s in a name?  Maybe more than you think!

 

· · · · · · ·

We are Rebecca Stewart, Chief Practice Officer, and Samantha Hagel, Chief Administrative Officer, both with The Improve Group, a firm based in St. Paul, Minnesota, with a mission to help mission-driven organizations make the most of information, navigate complexity, and ensure their investments of time and money lead to meaningful, sustained impact.

This year, we decided to consider a beautiful question: What is The Improve Group’s role in creating a diverse, inclusive field of evaluation? The question emerged as we were thinking about the 2015 Year of Evaluation and its focus on equity, on the AEA’s cultural competency statement, and our own desire to promote social justice. As we’ve pondered this question, we realize part of our role is to share lessons learned and success stories with others in the field of evaluation.  So, here goes!

Hot tip: One way we support diversity and inclusion in our practice is to utilize a competency model. Our competency model asks all of our staff to have a constant awareness of, be learning about, and apply cultural competence. It is not a one-time thing; we want to see team members thinking about this all the time, unprompted. We support them with several opportunities to reflect and learn from each other, including an organization-wide conversation on unconscious bias.

Lesson Learned: Develop strategies to diversify the pipeline of people entering the field of evaluation and applying for open positions. Over the years, the vast majority of our candidates have come from a single graduate program. This year, we are experimenting with connecting with non-traditional audiences. For example, we gave presentations in undergraduate programs, and to Vista and AmeriCorps members at Public Allies and the PSEI Vista Program, to raise awareness of evaluation as a profession. We also changed our internship program from a purely graduate-level program to a summer internship for undergraduates and a school-year partnership with the GEDI program.

Hot tip: Be expansive, curious, and collaborative in seeking out community partners. We got to know an organization, Partnership Resources, through an evaluation of how the Americans with Disabilities Act has affected employment for people with disabilities. This interaction challenged us to figure out how to be a supported employment site. We looked at our workplace in a new way and found a role for a new employee, matched to us by Partnership Resources.

Rad resource: We partnered with New Sector Alliance for our new summer internship program. We have a shared interest in broadening the scope of those entering the social sector, and they have helped us reach potential new evaluation professionals.

Interested in further conversation? Join us at the conference: http://bit.ly/1PbkqOV

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

Hello, I’m Jori Hall, assistant professor at the University of Georgia and a member of the AEA Public Statement on Cultural Competence in Evaluation Dissemination Working Group. This tip is focused on integrating cultural competence into everyday practice through values-engagement.

Tips:

  • As suggested in the AEA Public Statement on Cultural Competence in Evaluation, all evaluation practice and evaluands are situated in and influenced by cultural norms, values, and various ways of knowing. Values-engagement acknowledges these influences and attempts to be responsive to the dynamic interaction between the values reflected in evaluation practice and the evaluand. That is, values-engaged evaluators understand that evaluation practice promotes values, and that these values must respectfully engage stakeholders’ values.
  • Values-engagement is not a specific strategy or a set of required methods; rather, it is a commitment to culturally responsive evaluation. While there is more than one way to be values-engaged, the commitment to culturally responsive, values-engagement suggested here involves the evaluator prioritizing values of inclusion and equity in everyday practice. Inclusion refers to engaging and describing the plurality of stakeholders’ values, perspectives, and concerns, focusing on the least well served in a particular context. Equity refers to how well and to what extent the evaluand is attending to stakeholder groups (i.e., access, participation, etc.) in the context. Because values-engagement advocates inclusiveness and the equitable treatment of stakeholders, it supports the goals of the Statement on Cultural Competence in Evaluation.
  • Values-engagement can be integrated throughout the life cycle of an evaluation, and enacted through generating evaluation questions, data, and dialogues related to the ways in which the evaluand is attending to the cultural values of the groups represented in the context. To learn more about values-engagement, its connection to cultural competence, and how evaluators can practically enact its commitments in different evaluation contexts, begin with the resources provided below!

Rad resources:

The American Evaluation Association will be celebrating Cultural Competence Week. The contributions all this week come from the Cultural Competence committee. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluator.

· · · · · · · ·

My name is Jennifer Sullivan Sulewski and I am a Research Associate at the Institute for Community Inclusion (ICI) at the University of Massachusetts Boston. At ICI I have worked on a variety of research and evaluation projects related to services and supports for people with disabilities. I currently am most involved with Work Without Limits, a public-private partnership funded by the Massachusetts Medicaid Infrastructure and Comprehensive Employment Opportunities (MI-CEO) grant to strengthen the Massachusetts workforce and advance work opportunities for youth and adults with disabilities in Massachusetts.

Universal Design refers to “the design of products and environments to be usable by all people, to the greatest extent possible, without the need for adaptation or specialized design” (http://www.design.ncsu.edu/cud/about_ud/udprincipleshtmlformat.html). The principles of universal design can be applied to evaluation to ensure that all relevant populations are included at every stage of the work, from project design to sharing of findings.

Hot Tip: Universal design is helpful to think about even if your evaluation is not specifically focused on disability programs or issues. You will likely encounter people with disabilities or members of other vulnerable populations whatever your focus, so it is good to be prepared. Moreover, good universal design works better for everyone, even those without disabilities or other barriers. Think of bar patrons watching TV with closed captioning on, or people with strollers or rolling suitcases using elevators and curb cuts.

Hot Tip: One key aspect of applying universal design to evaluation work is to think about all the different ways people communicate or access information. For example, if you are conducting a survey on-line, you might offer the option to do it on paper or over the phone if respondents prefer not to respond by computer. Or if you are doing interviews by phone you may find that some respondents prefer to speak in person or respond by e-mail.

Rad Resource: For more tips, join the AEA Coffee Break Webinar on this topic on Thursday, August 5 at 2PM – click here to learn more and sign up. The Disability and Other Vulnerable Populations TIG is also hosting multiple sessions related to Universal Design, including a skill building workshop and a roundtable, at the upcoming American Evaluation Association Annual Conference in San Antonio the first week in November, so look for us there!

The American Evaluation Association is celebrating Disabilities and Other Vulnerable Populations (DOVP) Week with our colleagues in the DOVP AEA Topical Interest Group. The contributions all this week to aea365 come from our DOVP members and you may wish to consider subscribing to our weekly headlines and resources list where we’ll be highlighting DOVP resources.

· · ·

Archives

To top