Greetings! Caitlyn A. Bukaty here sharing some exciting insights with you during Disabilities and Underrepresented Populations Topical Interest Group week.
Today I want to offer a few ideas my fellow evaluators might find helpful in making their evaluations more accessible to a wide range of stakeholders. This information comes from my experience collecting feedback from young adults with intellectual disabilities who participated in a workplace problem-solving intervention, but one of my favorite features of these techniques is how helpful they are to a wide range of stakeholders! This concept is known as Universal Design, and the premise is that an option you might offer to one groups of stakeholders, for example those who have difficulty reading, actually makes accessing your evaluation materials easier for other groups, such as stakeholders for whom English is not their first language, or those with visual impairments.
Without further ado, let’s explore some ideas to help your evaluations reach for the stars in terms of accessibility!
Hot Tips:
- Add pictures – A well-connected photo can help stakeholders link a question to a certain event, or clarify a response.
In this example, a series of question are linked to a certain part of the intervention using a picture of the person with whom participants interacted:
Here responses are clarified with “thumbs up” and “thumbs down” symbols
- Go digital – Offering traditionally “print” materials in digital format opens up a universe of accessibility for stakeholders. Users can access screen reader software, text-to-speech features, and even translation applications to better understand the material. This is even more effective if materials are offered on a mobile friendly platform, mobile web access is widely reported as overtaking desktop computer use.
- Be all ears – Prepare to accept responses from your stakeholders in a variety of creative ways. Offering stakeholders multiple options for response may mean gathering responses from those who would not have been able to participate via a single mode of response. Written or typed responses to forced choice and open ended questions may be traditional, but what if someone wants to dictate a response…can you make a scribe available in person or via telephone to support his or her participation? How about a participant wishing to record a response? This can be achieved via a voice or video recorder on many mobile devices. Depending on the question a pictorial response, such as indicating time spent on a circle graph, might even encourage respondents to participate.
Rad Resources:
- Creative Commons Zero (CC0) Imagery – This is the name given to images free from copyrighting. In addition to taking or requesting photos specific to the topic of your evaluation there are resources linking you directly to CCO images such as Pixabay and Unsplash. Web search platforms, such as Google Images also allow you to specify reuse policies during an imae search.
- Web-based Survey Platforms – These are useful for creating digital surveys or questionaires. Many are mobile friendly, and several platform offer free features or use. Try SurveyMonkey, SurveyGizmo, QuestionPro, or Google Forms.
Lesson Learned:
- The idea behind today’s post is to maximize stakeholder participation by inviting them to take part in an evaluation in whatever way is most convenient and effective. To learn more about universal design geared towards materials development and response check out the Universal Design for Learning materials offered through CAST.
The American Evaluation Association is hosting the Disabilities and Underrepresented Populations TIG (DUP) Week. The contributions all week are focused on engaging DUP in your evaluation efforts. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hello Caitlyn,
Thank you for the great tips and the reminder that Universal Designs can and should be universally applied to all interactions with others – ability cannot be inferred!
A colleague and I have engaged in a debate and I was wondering what you think. It has to do with your third ‘Hot Tip’.
If you are creating different evaluations to meet different needs (e.g. one that works in Kurzweil, can be read by Dragon or uses images to help clarify the question) are you potentially skewing the data that you might get across all three formats of tests?
The reason this came up is that I work with students with disabilities at the college level and often have to read aloud or clarify an evaluation that is not accessible to my students, for example a key performance indicator survey. In my clarifying I am likely providing information other students might not have, or even projecting my own bias onto the survey.
I agree that evaluations NEED to be accessible, but how could we account for the potentially inaccurate results that may emerge from well meaning intervention?
Looking forward to reading your thoughts,
Kristen
Hi Caitlyn,
I really enjoyed this article, not only because it applies to the class I am currently taking, but also because of its implications for me as a teacher. Throughout my teaching degree, there were many classes that addressed universal design and its benefits in classroom and curricular development. As you say, universal design can be helpful to a wide range of stakeholders by making information more accessible and understandable from different points of view.
While universal design directly applies to educating students in a classroom, I think that it also has a lot to offer in communicating with peers and with student’s parents or guardians. So often we find ourselves choosing specific teaching methods, or using various strategies because we are aware of the various benefits, but teachers rarely explain these methods to parents. I wonder if teachers were more open about their strategies and choices, then parents would be less inclined to question those methods.
I found your Hot Tips very helpful, particularly the part about “going digital” and being open to all responses from stakeholders. I agree that people having mobile access to information makes a big difference in today’s technologically driven culture. People often find it difficult to accept criticism, be it constructive or not, but in evaluation it is so important because it helps evaluators avoid bias and remain on task.
Hi Caitlyn,
I really enjoyed reading your post on Skyrocketing your Accessibility. Being able to make evaluations more accessible to a wide range of stakeholders is critical in this day and age. When do evaluation it is crucial to get as much data to evaluate as possible. Allowing students who may have trouble providing the data or making it more accessible is the way of the future. We talk so much about creating an inclusive environment for students to learn especially in my school division. In my school division we have disassembled any programs that would segregate students that may have a learning disability, Fetal Alcohol Syndrome, behaviour problems, or low cognitive functioning. The struggle with inclusion is that we now need to look for other ways to allow them to express their thoughts, feelings, etc.
If we can enable students who have disabilities to participate in providing data to the evaluators and in turn a wide variety of stakeholders then they will feel like they are part of the process. The one thing that I can see as part of a downfall is that the students may only respond with what they think the evaluator would like to hear and not what they actually think. Although this may be more of a global response especially in the lower grades, how do we allow them to have their authentic voice heard and feel like it is confidential.
I appreciate the Universal Design concept and feel this would benefit students who are filling out the survey and also stakeholders. The more effort that it takes for a person to complete a survey the less likely they will do it or do it with the most accurate information. If we make things simple for everyone you will likely get a better response rate that is more accurate. As I have learned we want to collect data that is imperative to our program and purpose to analyze. This concept will enable everyone to give their input.
By allowing people to respond in numerous ways and making it accessible to everyone will maximize the data collected and allow for greater interpretation, effective and not effective ways, patterns and new interventions to be put in place to make the program more effective.
Thank you again for introducing me to this new technology. I look forward using in the future when I do my program evaluation next year.
Darla