Hello, my name is Michael Lambur. I am currently the Associate Director for Program Development with Virginia Cooperative Extension. I have been an Extension evaluator since 1985 and recently spent five years as the Evaluation and Research Leader with the eXtension initiative.
Website usability testing measures the suitability of a website for its users and is directed at measuring the effectiveness, efficiency, and satisfaction with which users can achieve specified tasks while using the site. Basically, a user is given a set of tasks to complete while being videotaped talking out loud about their experience. In my time with eXtension, we conducted two usability tests of the initiative’s public website. The first was done face-to-face by a contractor. It was well done, informative, and rather expensive. In the more recent one, we used an online usability testing service. Again, it was well done, informative, amazingly inexpensive, and very timely.
Lessons Learned:
Online usability testing services do work. We achieved essentially the same results using the online service versus face-to-face at a fraction of the cost. Our face-to-face usability testing cost $12,000 using 12 participants. Our online usability testing cost $280 using eight participants. In addition, the online service provided trained testers based on demographics we provided. And we received a video of the results in about an hour.
The key to usability testing is a set of tasks that reflects the purpose of the site. You need to develop a set of very specific tasks the user can move through that truly reflects how you want them to understand and use the site. For the online service, the tasks needed to be completed in 15 minutes, whereas the face-to-face lasted about 45 minutes. The quality of feedback received from the online testing was excellent—you can achieve a lot in 15 minutes.
Usability test results can be brutal. Be prepared to have your bubble burst when viewing the video of people using your website. What we intend in developing a site and what people experience in using it can be very different. While it typically isn’t all bad, the results are often eye-opening. Keep in mind that the end result is an improved website that best serves your users.
Rad Resources:
The two online services we used were:
The American Evaluation Association is celebrating Extension Education Evaluation (EEE) TIG Week with our colleagues in the EEE Topical Interest Group. The contributions all this week to aea365 come from our EEE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.