Hi, this is the final post for Government Evaluation TIG week! I am Elise Garvey and I’m a Senior Management Auditor with the King County Auditor’s Office in Seattle, Washington. So many services and important information provided by the agencies we evaluate have moved online. As evaluators, we can help assess whether those online resources actually meet the needs of the intended users through usability testing. This can provide valuable feedback about whether the resource is accessible, informative, easy to navigate, responsive to important needs, etc.
Hot Tips:
Here are some steps you will want to include in your usability testing:
- Identify usability testing tasks: create no more than ten discrete tasks you would like a person to be able to do.
- Example: You want someone who needs help with a concern find an assistance hotline number
- Define successful completion of a task: Identify a few metrics that indicate un/successful completion of a task. Metrics could be those that you observe or that a person could describe to you out loud.
- Examples:
- Whether the task was completed
- Whether the task was completed without error
- Level of difficulty the person had completing the task
- Amount of time spent doing the task
- Number of clicks it took to do the task
- Expectation test (ask them to gauge what they think will happen to do task, compare with what actually happens)
- Person’s like/dislikes/recommendations for the process
- Identify testers: Identify 6 to 12 testers to complete the tasks. Testers should be potential users of whatever site or resource you are testing. To the extent possible, try to identify people who are not already familiar with using the resource you’re testing. You may want to consider monetary or other incentives to encourage participation.
- Decide on how you will be collecting data: There are some options:
- Observe during testing- sit with the person while they are completing the tasks. You can engage and moderate (i.e. tell them what the task is, ask questions to gather information about metrics as the task is being done) or you can create a barrier and observe (i.e. behind a mirror or window and just observe).
- Do a retrospective interview- After the person is done testing, ask them questions about their experience.
- Examples:
Lesson Learned:
- Don’t forget to test your usability test before implementing! Sometimes it may seem like you’ve come up with a good plan and tasks, but testing helps you see any adjustments that can be made to improve your methods.
- Remind your tester that you are testing the resource, not their skills. Sometimes testers can feel they are failing if they can’t complete the task, so it is helpful to state upfront that you are not testing their personal skills or abilities.
Rad Resources: gov from USDHHS
The American Evaluation Association is celebrating Gov’t Eval TIG Week with our colleagues in the Government Evaluation Topical Interest Group. The contributions all this week to aea365 come from our Gov’t Eval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.