My name is Randahl Kirkendall. I am a public health manager turned evaluator. Platometrics is the name of my consulting business, which for the past three years has been focused on program research, planning, and evaluation. I am also a part-time evaluator for the Science Education Resource Center at Carleton College, which provides faculty professional development programs using a combination of workshops and web-based resources.
Four years ago while overseeing the development of two websites I learned how to use Google Analytics to track and measure website use. My first contract to evaluate website content was two years ago. Since then, I have learned much about evaluating program websites, but still consider myself to be on a steep learning curve in this area. Here is a little bit of what I have learned.
Lesson Learned: Using multiple and mixed evaluation methods that include both quantitative and qualitative metrics is the best way to fully understand the processes by which a website is being used as well as the outcomes that result. Web analytics can reveal much about how users navigate a website, which is something that users have difficulty recalling. Surveys and interviews can measure their motivations behind their website use, the impacts and outcomes of using a website, and descriptive information about the users themselves. Combining the two helps to provide a more complete picture that may also include the interplay between the website and other aspects of a program, such as a workshop or printed material.
Rad Resource: Occam’s Razor by Avinash Kaushik (www.kaushik.net/avinash). This website is built around a blog by an expert in web analytics who presents information in an easy to understand and good humored way. You might also want to check out his book, Web Analytics 2.0.
Hot Tip: I am currently developing a short Guide to Evaluating Program Websites, which I will post on www.platometrics.com later this month. If you would be interested in reviewing a draft or would like to be notified when it is posted, send me a note at rk@platometrics.com.
This is a relatively new and rapidly evolving area of evaluation, so if you know of any other good resources or ideas, please share them.
This contribution is from the aea365 Tip-a-Day alerts, by and for evaluators, from the American Evaluation Association. If you’d like to learn more from Randahl, consider attending his session at the AEA Annual Conference this November in San Antonio. Search the conference program to find Randahl’s session or any of over 600 to be presented.
Hi Randy,
Sorry for the delay–your follow-up comment fell through the cracks! Anyway, yes, Qualtrics’ heat map question is very cool. And this is exactly what the tool I mentioned does, but for an entire web page. The tool is Feng-GUI (feng-gui.com). I haven’t had a chance to play around with it much, but as you noted, it just opens up so many possibilities for real-time data capture!
Thanks Marcus. I would be interested to learn how you are or are planning to use measures of hot spots. I recently saw a demo of Qualtrics survey software than enables heat maps (which record the hot spots) to be incorporated into the survey. It send my mind going with the possibilities for real time data capture.
Randahl,
Great information! As you mentioned, using multiple and mixed methods is the best approach to evaluating websites. AEA staff employs a variety of methods, including those you mentioned, as well as those such as backlinks. Though, one tool I came across recently allows a website administrator to actually assess mouse movements to determine website “hot spots.” Cool stuff!