usability study of wal-mart website

1

Walmart.com Usability Study

By Robert Bialecki

Usability Design and Testing

May 5, 2011

2

Table of Contents

Executive Summary…………………………………………….....3

Introduction………………………………………………………..4

Target User Analysis…………………………………………....5-6

Participant Demographics…………………………………………6

Methodology………………………………………………………7

Results…………………………………………………………8-13

Recommendations……………………………………………14-16

Appendix……………………………………………………..17-25

3

Executive Summary

I recently conducted a usability test on www.walmart.com using five participants who had no

prior experience with the website.

For more information about the test, see the Introduction section (p. 4 ) and Methodology section

(p. 7 ).

Based on the comments of the test participants, I have compiled a list of recommendations for

enhancing the usability of www.walmart.com. The recommendations are listed in order of

importance, based on the results of the usability test.

1. Decrease the size of the on-screen pop-up that asks if users want a service plan when

they add a product to their shopping cart.

2. Make the “In-Store Events” link easier to find, possibly in the top navigation bar.

3. Turn the “New Customer?” text in the upper right hand corner of the page into a link.

4. Include an “Add to Cart” button on the pictures and brief descriptions in the search

results for products.

5. Reduce the number of images and ads on the homepage.

6. Turn the “Remove” link into a button, like the “Add to Cart” button.

For more information about the recommendations, see the Recommendations section (p. 14-16).

http://www.walmart.com/


4

Introduction

Wal-Mart’s website (http://www.walmart.com) was the focus of this usability project. Wal-Mart

is one of the largest retailers in the world. The main purposes of its website are to promote and

inform consumers about its products, to allow consumers the option of buying products online,

and to provide a forum for consumers to offer feedback and have their questions answered.

Because of the wide variety of products and services it has to offer, Wal-Mart’s primary

audience is typical American consumers. Their audience comes in both genders and many

different ethnicities and age groups. However, due to their reputation of having “everyday low

prices,” Wal-Mart is particularly popular among middle and lower class families.

The objectives of this usability test were to evaluate the usefulness and level of difficulty of four

common tasks that users of the website might do. Additionally, test participants pointed out

aspects of the website that made the tasks more difficult to complete.


5

Target User Analysis

Persona

Name: Kathy Johnson

Age: 35

Location: La Crosse, WI

Occupation: Stay-at-home Mom

Status: Married, 2 Children

Kathy Johnson is a 35-year-old stay-at-home mom who lives in La Crosse, Wisconsin. Her

husband, Karl, is an accountant at a local business. The couple has twin 10-year-old boys,

Tanner and Taylor, and the family lives in a modest suburban house.

Kathy has been a loyal Wal-Mart customer her whole life. As a young girl, she loved

accompanying her mother on the weekend shopping trip. Today, she recreates those memories

by taking her twin boys with her every weekend. As a stay-at-home mom, Kathy leads a very

busy life. Between maintaining a tidy household, getting the kids ready for school, and preparing

meals for her family, she also often travels to many nearby cities and towns, since her boys are

on the traveling soccer team. One thing Kathy has always loved about Wal-Mart is that she can

find clothes, groceries, toys, and much more all in one store.

Kathy has access to high-speed Internet because her husband must also work at home sometimes,

and the children need a computer for their homework. Kathy would estimate she spends about 6

hours per week using the Internet. In her leisure time, Kathy maintains a blog and keeps in touch

with friends from high school on Facebook and Twitter. Kathy also loves to shop online, where

she often finds better deals than those found in stores. Although she loves to shop both in-stores

and online, Kathy is very frugal, and Wal-Mart’s “everyday low prices” are what has kept her

coming back after all these years.

Key Characteristics: Conclusion:

Is a busy person She would like to find the information

she’s looking for as quickly as possible,

without having to deal with a lot of

clutter.

Shops Online She would be interested in buying Wal-

Mart products online

Often travels to nearby cities and towns She would like to know if there’s a

6

Wal-Mart in her travel destination.

Has access to high-speed Internet She won’t have trouble downloading

large files.

Uses social networking sites She might like to receive Wal-Mart

updates and information on those sites.

Likes to save money She would be interesting in knowing

about any coupons, discounts, deals,

and promotions that Wal-Mart has to

offer.

Demographics of the Participants in this Usability Study

Gender: 2 male, 3 female

Ages: 60, 55, 41, 31, 24

7

Methodology

For this study, I selected five participants who passed the screening process. Participants needed

to fit the following profile:

Be older than 18 years of age

Have more than one year of experience using the Internet

Be familiar with Wal-Mart

Have no negative opinions about Wal-Mart (the company or its practices)

Have never visited Walmart.com

The participants were asked to complete four common tasks that new users of the website might

do. For the purpose of consistency, each participant in the study took the test in the same

makeshift usability lab. The lab consisted of a room with: a desk, a computer with high-speed

Internet access, a comfortable chair, and adequate lighting.

I monitored the usability test for each participant and took notes on their procedures and thoughts

as they completed each task. The tasks were assigned in a different order for each participant so

that the participants’ gradual experience with the website didn’t skew the results of the study.

Due to resource limitations, the test sessions were not recorded on audio or video.

Participants were asked to complete a pre-test questionnaire, post-task questions after each task,

and a post-test questionnaire.

8

Results

Table 1 lists usability problems that participants encountered during the completion of the

usability test. In addition, the table lists the number of participants who experienced the

problems, participant comments in regard to the problems, and the severity level of the problem.

If 4 or 5 of the participants experienced a particular problem, it’s rated high severity. If 3

participants experienced a particular problem, it’s rated medium severity. If 1 or 2 participants

experienced a problem, it’s rated low severity.

Table 1

Usability Problems

Usability Problem

Number of

Participants who

Experienced

Problem

Participant

Comments Severity

A pop-up ad for

adding a service plan

appeared after

participants added an

item to their cart.

Participants clicked

“Close” rather than

“Continue,” which

required scrolling

down to find.

3 out of 3*

“Do I want to add a

service plan? No.”

(closes pop-up)

“Okay… so did it add

the item to my cart or

not?” “Oh, I guess it

must have because the

„My Cart‟ tab now

says 1.”

High

Participants had

trouble finding In-

Store Events

3 out of 5

“The link is too small

and inconspicuous.”

- Participant 2

“I probably would

have found it [the link

on the home page]

sooner if there

weren‟t so many

pictures distracting

me.”

Medium

“New Customer?” on

the home page is not a

link that allows you to

create an account.

3 out of 5

“I assumed clicking

on „New Customer‟

would take me

directly to the

registration. I thought

„Sign in‟ was for

established

customers.”

Medium

9

There is no “Add to

Cart” button on the

pictures and brief

descriptions in the

search results for

products. Participants

had to click on a

particular item in

order to get the “Add

to Cart” button.

3 out of 5 “Where‟s the „Add to

Cart‟ button?” Medium

There are too many

images and ads on the

homepage.

2 out of 5

“I don‟t like how you

have to scroll down

pretty far to find a lot

of the important

information at the

bottom of the page.”

Low

The “Remove” link is

harder to find than the

“Add to Cart” button.

2 out of 5 “That link doesn‟t

stand out very well.” Low

After entering an

address to get driving

directions to a store,

participants saw a

map but had to scroll

down to get the

written directions.

1 out of 5

“Where‟s the written

instructions? … Oh,

there they are.”

Low

* The service plan pop-up appeared only for users who searched for major appliances and

electronics in Task 3.

10

Tables 2 through 5 show the ratings participants gave for the difficulty level of the task, the

amount of time it took to complete the task, and the likelihood that they would use each

task/feature.

Table 2

Task 1: Create a user account at www.walmart.com.

Difficulty Level of Completing This Task Very Easy

Somewhat

Easy

Neither Easy

nor Difficult

Somewhat

Difficult Very Difficult

Number of

participants

who gave this

rating

2 3

The Amount of Time it Took to Complete This Task

A Lot Shorter

Than

Expected

Somewhat

Shorter Than

Expected

About the

Amount of

Time

Expected

Somewhat

Longer Than

Expected

A Lot Longer

Than

Expected

Number of

participants

who gave this

rating

4 1

The Likelihood That They Would Use This Task

Very Likely Somewhat

Likely

Neither

Likely nor

Unlikely

Somewhat

Unlikely

Very Unlikely

Number of

participants

who gave this

rating

1 2 1 1


11

Table 3 Task 2: Get driving directions from your home to the nearest Wal-Mart store.


Somewhat

Easy

Neither Easy

nor Difficult

Somewhat


Number of

participants

who gave this

rating

1 1 2 1


A Lot Shorter

Than

Expected

Somewhat

Shorter Than

Expected

About the

Amount of

Time

Expected

Somewhat

Longer Than

Expected

A Lot Longer

Than

Expected

Number of

participants

who gave this

rating

1 2 1 1



Likely

Neither

Likely nor

Unlikely

Somewhat

Unlikely

Very Unlikely

Number of

participants

who gave this

rating

2 2 1

12

Table 4 Task 3: Add a product of your choosing to your online shopping cart and then

remove it from your shopping cart


Somewhat

Easy

Neither Easy

nor Difficult

Somewhat


Number of

participants

who gave this

rating

1 2 2


A Lot Shorter

Than

Expected

Somewhat

Shorter Than

Expected

About the

Amount of

Time

Expected

Somewhat

Longer Than

Expected

A Lot Longer

Than

Expected

Number of

participants

who gave this

rating

1 2 2



Likely

Neither

Likely nor

Unlikely

Somewhat

Unlikely

Very Unlikely

Number of

participants

who gave this

rating

1 1 2 1

13

Table 5 Task 4: Find a list of events taking place at your nearest Wal-Mart within the

next two months.


Somewhat

Easy

Neither Easy

nor Difficult

Somewhat


Number of

participants

who gave this

rating

1 2 1 1


A Lot Shorter

Than

Expected

Somewhat

Shorter Than

Expected

About the

Amount of

Time

Expected

Somewhat

Longer Than

Expected

A Lot Longer

Than

Expected

Number of

participants

who gave this

rating

2 2 1



Likely

Neither

Likely nor

Unlikely

Somewhat

Unlikely

Very Unlikely

Number of

participants

who gave this

rating

1 2 2

14

Recommendations

Based on the comments of the test participants, I have compiled a list of recommendations for

enhancing the usability of www.walmart.com. The recommendations are listed in order of

importance, based on the results of the usability test.

1. Decrease the size of the on-screen pop-up that asks if users want a service plan when they

add a product to their shopping cart.

Reason: After selecting “Yes” or “No,” users have to scroll down to see the “Continue”

link. The test participants all clicked on the “Close” button in the upper right hand corner

and weren’t sure if the product was added to their cart or not.

2. Make the “In-Store Events” link easier to find, possibly in the top navigation bar.

Reason: Test participants had a difficult time finding the link, as it required them to

scroll half-way down the homepage and sort through less relevant content.

3. Turn the “New Customer?” text in the upper right hand corner of the page into a link.

Reason: New users currently have to click on “Sign In” to create an account, which

implies that the use already has an account. (See Figure 1)

Figure 1


15

4. Include an “Add to Cart” button on the pictures and brief descriptions in the search

results for products.

Reason: Test participants had to click on a particular item in order to be taken to a screen

with an “Add to Cart” button. Some wondered why the button wasn’t already in the

search results. (See Figure 2) This change would be useful for users who already know

exactly what they’re looking for.

Figure 2

5. Reduce the number of images and ads on the homepage.

Reason: The homepage requires quite a bit of scrolling to get from the top to the bottom

of the page. Since there is relevant content throughout the page, this would make that

content easier to find.

16

6. Turn the “Remove” link into a button, like the “Add to Cart” button.

Reason: Some test participants found the “Remove” link hard to find because the text is

small and it’s surrounded by other links. (See Figure 3)

Figure 3

17

Appendix: Usability Protocol

General Script

The screening process:

“Excuse me, could I have a moment of your time?”

If no, “Okay then, sorry to bother you.”

If yes, “Hi, my name is Robert Bialecki. I‟m a student at the University of Wisconsin-Stout, and

I‟m conducting a study for my Usability Testing class. Would you be interested in participating

in my study?”

If no, “Okay then, sorry to bother you.”

If yes, “Okay, thank you. First, I‟d like to ask you a few questions to see if you would qualify for

the study.”

(Ask the questions from the Screener Form and record the responses)

“That‟s all the questions I have for you. Thank you! If you are selected to participate in this

study, what is the best way for me to contact you to schedule a session?”

Before the test:

“Hello, and welcome to the usability study. Again, my name is Robert Bialecki. If you need a

beverage or a restroom, they‟re right over there. As you know, I‟m conducting this study for my

Usability Testing course at UW-Stout. The purpose of my study is to evaluate the Wal-Mart

website. Specifically, I will ask you to complete a series of common tasks on the Wal-Mart

website that new users might use. This usability test will help me evaluate the ease and

usefulness of the tasks. Before we begin, I‟d like to go over this consent form with you. If you

have any questions or concerns regarding the consent form or the study itself, please let me

know right away.”

(Go over the form with the participant, allowing him/her time to read and sign)

“So do you have any questions or concerns?”

“Okay, everything looks good. I would also like you to fill out this questionnaire before we

begin.”

(Allow participant time to complete the questionnaire.)

“Okay, looks good.”

The test:

“So here‟s how the test will work - I will give you four tasks to complete on the Wal-Mart

website. Each task will require you to start at the home page you see currently displayed on the

screen. I will be beside you as you complete each task, but I will not be able to tell you how to

complete the tasks. I will be taking notes on your procedures, thoughts, and my own observations

as you complete each task. There is no time limit on completion of the tasks, but if at any point

you decide a task is too difficult, you may give up, and we will move on to the next task. I would

also like to encourage you to think out-loud as you complete each task. Say out-loud what‟s on

your mind as you complete each task. The thoughts could be about anything related to the task or

the website. For example, you might say „Okay, so next I‟m going to…‟ or „I think it would be

easier if…‟ or „This is frustrating because…‟ I know the process of thinking out-loud may feel

18

weird at first, but your thoughts will be very beneficial to my study. Do you have any questions at

this point? Okay then, here is your first task…”

After the each task:

“You‟re doing a great job. Again, please remember to think out-loud. Do you have any questions

so far? Okay, now before we move on to the next task, I‟d like you to fill out this short

questionnaire about the task you just completed.”

(Allow time for the participant to read and fill out the questionnaire)

“Great! Thank you! Now let‟s move on to the next task. Please return to the home page. Your

next task is…”

After completing the test:

“Well, that‟s the last task. You‟ve done a very good job, both with the tasks and thinking out-

loud. There‟s one last thing I‟d like you to do. Please fill out this questionnaire about the test

overall.”

(Allow time for the participant to read and fill out the questionnaire)

“Okay, that‟s all I have for you. Do you have any last questions? Again, I‟d like to thank you for

your time and participation in this study. Have a nice day!”

19

Screening Form

Candidate Name: ______________________ Qualifies? □ Yes □ No

Screening Questions:

1. Are you older than 18 years of age?

□ Yes (continue) □ No (terminate)

2. Do you have more than one year of experience using the Internet?


3. Are you familiar with the company, Wal-Mart?


4. Do you have any negative opinions about Wal-Mart (the company or its practices)?

□ Yes (terminate) □ No (continue)

5. Have you ever visited Wal-Mart’s website, Walmart.com?

□ Yes (terminate) □ No (continue)

Cell Phone # _______________________

Home Phone # ______________________

E-mail _____________________________

20

Consent to Participate in Wal-Mart Website Usability Study

Title: Walmart.com Usability Study

Moderator: Robert Bialecki

Description:

This is a usability study of Wal-Mart’s website, Walmart.com. The purpose of this study is to

evaluate the effectiveness of the website in order to help better meet the needs of Wal-Mart’s

consumers. You will be asked to complete several tasks that users of the website would

commonly do, as well as answer some questions that pertain to your experience while interacting

with Walmart.com. You will be observed only by the moderator who will be taking notes;

however, you will not be recorded by video or audio.

Time Commitment:

Your involvement in this study will take no longer than one hour.

Risks:

There are no adverse risks involved in the participation of this study. This study is solely for

educational purposes in the completion of a college course project.

Confidentiality:

Your involvement in this study will be held strictly confidential by the moderator. No potentially

identifiable information will be shared with anyone for any reason in any aspect of this study.

Right to Withdraw:

Participation in this study is strictly voluntary. There will be no consequences should you choose

not to participate in this study. Furthermore, you may withdraw from this study at any time and

for any reason, even if you have already agreed to participate.

Moderator

Robert Bialecki

University or Wisconsin-Stout Student

(608) 783-0529

[email protected]

Statement of Consent:

By signing this form, you acknowledge that you have read and understood the information above

and that you agree to participate in the Walmart.com Usability Study.

_______________________________________________ __________________

Signature Date

mailto:[email protected]

21

Pre-test Questionnaire

1. Have you ever visited the website of any retail store before?

□ Yes

□ No

If you answered “Yes,” please answer the following two questions:

Which site(s) did you visit?

Are there any aspects in particular that you liked or disliked about the site(s)?

2. Have you ever purchased anything online from a retail store before?

□ Yes

□ No

If you answered “Yes,” please answer the following two questions:

Which retailer(s) did you purchase from?

Was your experience a positive one? Why or why not?

3. What sorts of information do you feel should be available on a retailer’s website? Please list

what you feel is most important.

4. What sorts of features do you feel should be available on a retailer’s website? Please list at

what you feel is most important.

22

Tasks

1. Create a user account at www.walmart.com

2. Get driving directions from your home to the nearest Wal-

Mart store.

3. Add a product of your choosing to your online shopping cart

and then remove it from your shopping cart.

4. Find a list of events taking place in the next two months at

your nearest Wal-Mart store.


23

Post-task Questionnaire (same for each task)

1. Rate the difficulty level of completing this task:

Very Easy Somewhat Easy Neither Easy

nor Difficult

Somewhat


Please explain your rating:

2. Rate the amount of time it took to you to complete this task:

A Lot Shorter

Than I

Expected

Somewhat

Shorter Than I

Expected

About The

Amount of Time

I Expected

Somewhat

Longer Than I

Expected

A Lot Longer

Than I

Expected


3. Rate the likelihood that you would use this feature/task:


Likely

Neither Likely

nor Unlikely

Somewhat

Unlikely Very Unlikely


How might this task be made easier?

24

Post-test Questionnaire

1. Overall, rate the average difficulty level of the tasks:

Very Easy Somewhat Easy Neither Easy

nor Difficult

Somewhat



2. Rate your overall satisfaction with the Wal-Mart website:

Very Satisfied Somewhat

Satisfied

Neither

Satisfied nor

Dissatisfied

Somewhat

Dissatisfied

Very

Dissatisfied


3. Would you recommend Walmart.com to friends or relatives if they asked? Why or why not?

25

Checklist

Before participant arrives:

□ Clear the desk of any unnecessary clutter or potential distractions

□ Access the Internet and go to the Walmart.com home page

□ Make sure a pen is available for questionnaire completion

□ Have pen and paper ready for note taking

Welcome:

□ Introduce myself and thank participant for participating

□ Show participant where to find beverages and a bathroom

□ Escort participant to the testing room

Before the test:

□ Explain the purpose of the test.

□ Review the Consent Form with the participant, allow time for him/her to read and sign

□ Ask the participant if he/she has any questions or concerns

□ Give the pre-test questionnaire

Instructions:

□ Explain the procedure of the test

□ Ask participant to think out-loud during test; explain what it is and give examples

After each task:

□ Offer reassurance if participant seems to have had difficulty with task

□ Remind participant to think out-loud, if necessary

□ Give post-task questionnaire

□ Ask participant to return to the Walmart.com home page

End of test:

□ Give post-test questionnaire

□ Thank participant for participating

usability study of wal-mart website

Documents