AEA365 | A Tip-a-Day by and for Evaluators

TAG | website

Hello! I am Carey Tisdal, Director of Tisdal Consulting in St. Louis, Missouri. I work with people who develop informal learning experiences for museum exhibitions, museum programs, documentary films, and media-based projects. Many of my projects include websites as one element of a learning system. I used the Building Informal Science Education (BISE) project as an opportunity to develop a framework to focus studies involving websites. This experience helped me improve my own practice by analyzing other evaluators’ work as well as connecting to key concepts in the website evaluation literature. I hope you find it useful, too!

I developed my website evaluation framework by analyzing 22 reports from the BISE database that were coded as “website” evaluands (i.e. the entity being evaluated). The overarching method I used to analyze the reports was Glaser & Strauss’ Grounded Theory. I then connected concepts in the program theory to literature about website evaluation. The resulting website evaluation framework uses high-level program theory to guide the identification of focus areas and questions to structure website evaluations. As illustrated in the graphic below, I organized seven of the major areas of consideration as a set of sequential, necessary steps influencing User Impacts and System Effectiveness. Read my whitepaper, “Websites: A guiding framework for focusing website evaluations,” to learn more!

Tisdal

Lessons Learned:

  • Some of the evaluations I reviewed focused on appeal (content, visuals, or forms of engagement), which is certainly a very important aspect of website evaluation. Yet, when connecting the focus areas, I realized that without testing usability, as well as appeal, it is not possible to draw strong conclusions about how audience impact is or is not accomplished.
  • Evaluating the system effectiveness of a website is essential in multiplatform projects. Awareness and access play important roles in whether or not users of other parts of an informal education system (e.g. an exhibition, program, or film) even get to the website, or, in turn, if website viewers see a film or attend an exhibition.
  • In my own work, I’ve found that this website framework helps project teams and website designers to clarify what they really need to know.

Rad Resources:

  • The U.S. Department of Health and Human Services offers an amazing set of resources to get you started in usability testing for websites. This site has been updated since I did my research and is now even better!
  • The BISE database and the website org provide access to a wide range of evaluation reports. When I need to look at how colleagues approached evaluation designs, they are my first stops!

The American Evaluation Association is celebrating Building Informal Science Education (BISE) project week. The contributions all this week to aea365 come from members of the BISE project team. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

We’re Amy Schaller and Bryna Koch, Evaluation Specialists at the University of Arizona, Cooperative Extension Services.

How do you support an inaugural cross-site evaluation with a national initiative while incorporating technology and building grantee capacity?  This was the charge of the CYFERnet (Children, Youth, and Families Evaluation & Research Network) Evaluation Team at the University of Arizona.  Our vision was to create a website that would serve as a central platform to house a range of evaluation resources, tools, and training materials for grantees.  And so, CYFERnetSEARCH.org was born.  The evaluation capacity-building website, designed to support the efforts of the Children, Youth, and Families At-Risk (CYFAR) initiative funded by USDA-NIFA, is also available to the public.

The resources we developed for the site included: content for eight learning modules including quizzes and videos, a searchable database of vetted evaluation instruments, a user-account feature, and online logic model and survey builders.  All of these features were developed from the ground-up and much energy was invested in generating original content, design, and features; no small undertaking. The process was, and remains, highly collaborative with the features on the website evolving numerous times since they were first developed.

Lessons Learned – DIY web development is fairly common these days, but to do so successfully requires some strategic vision and input from outside sources.  Here are some “Behind the Scenes” tips we can share from our experience to help those of you whose evaluation work intersects with website development:

  • If you can, work with web-design professionals.  Working across disciplines allows fresh thinking and combined perspectives. Solicit input from professionals with varying backgrounds.
  • Do your design research.  Select websites you like to use and consider what makes those sites appealing to you. Is it the feel or the look? How do they provide information? How do they make that information accessible?
  • Choose the tone for your site.  What feeling do you want to convey to your user?  What colors, images, and features communicate the essence of your project/company?
  • Do your platform research. Drupal, WordPress, Joomla? Should you consider open-source? How much modification is needed so the platform works for you?
  • Be the user. Interact with your site as a user would on a daily basis.
  • Organization and presentation matter. How can the content be intuitively grouped into themes that enable easier navigation for your users?
  • Consider the pace of technology and industry.  Will your design, platform and content be relevant in one or two years?

Rad Resources – 

cyfernetsearch.org – Our website

Trello.com A free online collaboration tool for project management

The American Evaluation Association is celebrating Arizona Evaluation Network (AZENet) Affiliate Week with our colleagues in the AZENet AEA Affiliate. The contributions all this week to aea365 come from our AZE members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · · · · ·

Archives

To top