Welcome to Day 3 of Inclusive Evaluation Design hosted by the Disabilities and Underrepresented Populations TIG. We are June Gothberg and Caitlyn Bukaty, the chairs of the TIG. This week’s theme centers around broadening opportunities for all individuals to actively participate in the evaluation process. We encourage evaluators to use Universal Design practices to improve accessibility and full participation in all phases of program evaluation from inception to dissemination. Today we are highlighting Principle 3 of Universal Design: Simple and Intuitive.
Hot Tips
Simplicity and intuitive design can help streamline evaluation participation for many users. Simple, intuitive design means content is easy to understand, regardless of the user’s experience, knowledge, language skills, or current concentration level. Your design will be most accessible if
- It eliminates unnecessary complexity.
- It is consistent with user expectations and intuition.
- It accommodates a wide range of literacy and language skills.
- It arranges information consistent with its importance.
- It provides effective prompting and feedback during and after task completion.
The cooking instructions depicted below show the difference between exclusively text-based instructions and instructions with text and graphics. The box on the right depicts a simple 3-step process for cooking oatmeal. In addition to succinct text directions, users have access to pictures demonstrating the prescribed actions for each step. The container on the left includes text-only directions.
Applying the same principle to evaluation data collection, below are three satisfaction surveys for which users can rate their satisfaction on a multiple-point Likert-type scale. The first uses a seven-point scale using the numbers 1-7 and text-only with no accompanying visual. The second uses cartoon emoticon faces showing a five-point scales with visuals matching the emotion and text. The final survey uses faces showing a six-point scale of emotion. While the third is clearly the simplest, let’s consider its intuitiveness. The third emotion options with no accompanying text depict unclear, possibly interchangeable feelings, which could lead to confusion. The most accessible survey here is Scale 2. With a limited range of response choices, using both text and visuals with a word that matches the depiction, this survey would be accessible to the widest range of users.
Lessons Learned
It’s important to check for reading level of all documents used in your evaluations including the evaluation plan, materials, instruments, and reports. Best practice suggests to keep reading levels at Grade 4 or lower to ensure materials are accessible to the widest range of users.
Rad Resources
- Microsoft Support (Free) for Readability – Microsoft provides readability statistics for most of its products. You can choose from ease of readability or grade-level. Each readability test bases its rating on the average number of syllables per word and words per sentence. For more information on the scores or how to set up readability in different languages for Word visit https://bit.ly/MSReadingLevel
- Universal Design for Evaluation Checklist (5th Ed) Principle 3 – this research-based checklist was developed specifically for evaluators by the DUP TIG at AEA to assist evaluators in designing inclusive evaluations.
The American Evaluation Association is hosting the Disabilities & Underrepresented Populations TIG (DUP) Week. The contributions all week come from DUP members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org . AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.