Seattle Art Museum Website
Usability testing is a valuable practice for observing how users interact with a system. It involves systematically observing user interactions under a controlled set of conditions.
For this project, we conducted a usability study of the Seattle Art Museum (SAM) website. Our goal was to assess the overall ease of use of seattleartmuseum.org. By testing key tasks with our target user group, we were able to identify major usability issues and provide prioritized recommendations for improving the overall user experience of the site.
Our objective for this study was to asses the overall usability of seattleartmuseum.org. Specifically, we wanted to determine how easily users completed key tasks by defining success metrics and identifying any obstacles that prevented task completion. We aimed to gather a combination of observational and subjective data in order to understand the overall ease of use and effectiveness of the site.
Since we had limited data on the site’s primary user group, we started with the assumption that our target users were art enthusiasts in the Seattle area.
I worked alongside three classmates on this research project. I was in charge of participant recruitment and coordination for our testing sessions. As a team, we all shared in the planning, execution, and data analysis stages, producing a research report and presentation as our final deliverable.
Tools: Morae, Tableau, Exce
Timeline: January 2017 – March 2017
Type: Class Project
We started by conducting a usability inspection of the website using 2 techniques: a heuristic evaluation and a cognitive walkthrough. These are both quick and low-cost diagnostic tools to help us identify and prioritize usability concerns before involving actual users.
For the heuristic evaluation, we used Nielsen’s 10 Usability Heuristics to guide our analysis:
We also conducted a cognitive walkthrough to understand key tasks, contexts of use, and both ideal and error pathways for completing these tasks. The 5 key task pathways are illustrated below:
Usability Testing Kit
In order to ensure the consistency of each testing session, we created a usability testing kit that would serve as a guide for both moderator and observers. We crafted 3 different scenarios, encompassing a total of 5 tasks. We identified the objective metrics we wanted to measure, as well as the method of collection for each metric. We also assembled a pre- and post-study questionnaire, as well as post-task questionnaires in order to gather subjective data from our participants.
The following spreadsheet served as a reference for our session note-takers:
We used the System Usability Scale (SUS) to measure subjective ease of use, including perceived effectiveness and satisfaction with the system:
We recruited participants by distributing a questionnaire to our networks, asking about demographics, interest in art, and experience purchasing tickets and making donations online.
During the testing session, participants were presented with each scenario and asked to complete all 5 task using a think aloud protocol. One team member served as the moderator, while the rest of the team observed remotely from a different room.
We collected data on task completion, ease-of-use rating, error rates, clicks per task, mouse movements, SUS survey responses, and qualitative responses. We also quantified the ease of use of the general system by averaging SUS scores across participants.
Samples of the data we collected are shown below:
For our final deliverable, we translated our findings into actionable design requirements: