IMDb Usability Study
10 weeks, Autumn 2016, team of four
In my Usability Research Techniques class, I worked in a team of four to analyze the IMDb website by designing a usability study, carrying it out, and presenting our design recommendations based on findings.
IMDb is an online database of information for movies and TV, including information on casts, plots, trailers, filming locations, entertainment news, and reviews.
We began proposing the study by outlining the product, client, client goals, target user group, some of the key tasks performed on IMDb.com, and potential usability questions.
Next, each member of the team performed an individual heuristic evaluation of one key task: finding the cast of a horror movie. From my evaluation, I found that IMDb has an incredible amount of content, but the content labels and categorization are neither helpful nor intuitive. This makes finding a specific piece of information like finding a needle in a haystack.
Next, we shared our individual heuristic evaluations as a team and identified issues in consistency, sense of navigation, and help & documentation.
Can users easily search for certain features, such as movies, genres, and actors, on IMDb’s website?
Can users easily navigate to and access different movie pages?
Can users easily understand the information that is presented on IMDb’s website?
How intuitive is the categorization of the IMDb website?
Study Plan & Recruitment
We planned for each study session to follow the same plan, with a script for consistency. Each session went as follows:
- Introduction and consent form
- Pre-test questionnaire
- Tasks, each followed by a short post-task questionnaire
- Post-test questionnaire
Each session took about 45 minutes to 1 hour and took place in a private conference room. During each session, one group member facilitated, while one or two others took notes. We used Lookback to record the screen of the participant's computer, and Google Hangouts to share the screen live with the notetakers. Each group member took a turn facilitating and note-taking.
We recruited participants through a screening survey (for ideal behavioral demographics) shared on various Facebook groups. We successfully carried out five study sessions.
We decided to learn more about our research questions by designing four tasks for participants to attempt during study sessions. We phrased each task as a scenario, so participants approach the task more naturally as they would in real life.
- It’s Thanksgiving night, and you and your family want to stream a comedy movie on your laptop, but cannot decide which movie to watch. Find out what kinds of comedy movies there are on IMDb.
On Halloween, you and your friend watched “The Conjuring 2.” You want to find another movie to watch and remember that you really enjoyed the actress who played the daughter, Judy, in the movie. Find another movie with that actress in it.
You heard one of your friends talking excitedly about a new movie called “Arrival.” You think it sounds interesting. Try to look it up yourself to see what’s so good about it.
Think of a movie that you recently saw. Find a movie similar to it in order to help you decide on what movie to watch next.
Through these four tasks, we aimed to observe how participants utilize the search function, how specific genres can be found, and how users generally navigate the website. We were also open to discovering new problems not yet identified.
We learned about our participants through the initial screening survey as well as the pre-test questionnaire. Our participants had been selected based on age range, gender (looking for a mix), frequency of IMDb use, and when they last used IMDb. We wanted infrequent users who had not been on the site lately. When selecting participants for the study, we wanted to find one completely new user, but we could not find any. This shows that most users in the selected age group have been on IMDb at least once.
Reasons for IMDb Use
To tackle the large amounts of data we collected during five study sessions, my team met for an affinity mapping/diagramming session. From this, we identified several themes across participants.
We identified multiple types of data from our study sessions to analyze: paths to task completion, participant comments during the study, where participants tried to click, errors, task success, frustration levels, and behavioral habits surrounding IMDb use. Using this, we identified three major findings.
Our design recommendations are based off of making changes to the website so that it works in the ways people expect it to.
Participants don't know how to find a specific genre.
We asked participants to find a comedy movie, but every participant had a hard time finding a list of comedy movies.
As seen in the image above, when a participant searches "comedy," a list of movies containing the word "comedy" appear as search results. These movies aren't necessarily in the comedy genre. Another way participants tried to find a genre list was by looking at the dropdown menu under "Movies, TV, and Showtimes." These are the two most common ways participants tried but could not find a genre page or list.
Design recommendation: Add genre pages/lists as a search result when a particular genre is searched. Add a genre page option under "Movies, TV, and Showtimes."
Participants are confused about what is clickable and what is not.
We watched several participants try to click on an image, expect it to link somewhere, and wait for the page to change, when the image was never a link.
Participants expected clicking on the highlighted smaller Despicable Me image, in the upper right hand corner of the movie image grid, to lead to the Despicable Me movie page. However, clicking on it does nothing. Hovering over the image shows a preview on the right.
The video below, from one of our testing sessions, illustrates this. Turn on captions to better understand what is being said.
Design recommendation: Because people are clicking it, the smaller image should link to the movie page.
Participants think the amount of content does not reflect the amount of content desired.
Many of our participants mentioned that there was too much content to sift through, and much of the information presented was not what they were interested in. We found this was especially true on the homepage, imdb.com.
Some participants even mentioned that the extra content made it difficult to find the information they really wanted. While this could be because our tasks only related to a small (but important) part of the site, we believe it still has a major impact on usability and is important to address.
Design Recommendation: Carry out another study, to learn about what IMDb users actually want on the website. Track what people search and click on. Prioritize the information preferred, and eliminate unwanted information or have its lower priority reflected properly.