wireframe testing and card sorting

Download Wireframe testing and card sorting

Post on 16-Apr-2017




0 download

Embed Size (px)


Paper Prototype Testing Card Sorting Webpage Comparisons

User TestingWebpage Comparisons Card Sorting

After Deployment / Guided NavigationMedically Ready ForceReady Medical Force1

My job with National Center for Telehealth and Technology. On a military base, working with Service Members and their Care Providers.

After Deployment website resource for SM concerned about overseas/deployment experiences.

Guided Navigation mobile tool meant to connect SM with relevant content

Three activities: user testing with paper prototypes, card sorting of webpage categories, and qualitative feedback for three proposed webpage designs1

2After Deployment / Guided Navigation

After deployment website. Topics of issues are given in a grid, with assessments and other resources in the menu2


A vs. B

Guided navigation wireframes designed by our UX Designer.

Design A on left with menu options on top of screen with collapsible categories.

Design B has two screens. First screen is the menu options, Second with expanded categories.3

User Testing Explosive Ordnance Disposal18 ParticipantsLt. SSgt. C+16 members from 3D EOD BN


18 total participants from EOD Battalion, a very young bomb disposal squad

Point of contact: Lt. S4

User Testing5# ParticipantsTest4Wireframes A, Web Comparison5Wireframes B, Web Comparison4Closed-Card Sort5Open-Card Sort

They were divided among the three research activities:

User testing with 2 wireframes, open and closed card sorting, and the website design comparisons5


A vs. B

We wanted to see if young soldiers, with no deployment experience, would explore the Curious level of interest

We also wanted to know if the categories made sense, and if the results were expected / helpful.6

Off-site testing with prototypesUser Testing

Testing was carried out on site, at our North Fort. We were permitted to only use the paper prototypes of our design7

In the following scenario, youre a newly recruited active duty soldier.Youve heard about soldiers exercising less when returning home; youre seeking information...

User Testing

A scenario was given related to their experience level.8

Feedback for design featuresDoes the layout make sense?User Testing

Feedback was recorded during this process9

User Feedback Wireframes A102/4 users skipped I am curious4/4 said the categories make sense1/4 commented on the Reset buttonReset would clear everything?

Out of four users tested with first wirefram, 2 skipped the curious option, a required first step.

When they were told, nothing seems to happen they were able to identify I am curious up top.

All four agreed with the category terms. One commented on the bottom Reset button, not sure if it would clear the check boxes or start all over10

Explore functionalityAble to find information?User Testing

User Feedback Wireframes B125/5 users selected I am curious

3/5 commented on I need helpA personal problem?Help and concerned are similarIf I were concerned, I would also need help

For wireframe Bs first screen, all five users identified I am curious

Three commented on the other options, suggesting a re-wording or combining them12

User Feedback Wireframes B135/5 said the categories make sense3/5 commented on the Reset / Results buttonsReset would start all over?Results is confusing from the past? Averages?Id make these options brighter to pop out

All five users navigated the categories successfully.

Three commented on the reset/results buttons, not sure about what theyre for, and suggested they stand out more.13

User Feedback Results Screen14

9/9 users liked the results overallGood informationI think theyre usefulResults are what was expected5/9 users liked the VideosId click on videos first, then read what theyre aboutId like to see more videos, dont like to read a lotId watch the video we get surveyed constantly

For our stretched-out results page, all nine responded positively to the results content.

Five showed a preference for videos14

Recommendations15Selecting a Level of Interest

When the user was prompted to select a level of interest (Curious, Concerned or Need Help) Design B was most successful

Based on user testing results these are the design recommendations:

Based on our findings, Design B is recommended for its level of interest screen15


Both designs were successful for category selectionsCombining the drop-down menus with color icons is recommended

Design ADesign B+=

For the categories section, Option A and B both did well;

Users responded positively to the drop-down menu and icons. A proposed combination of features is given here.

We were unable to test swiping down to search for additional categories, which could be an issue with the expanded categories of Option B.16

Recommendations17Reset / Results Bar

Users seemed confused with the terms Reset and Results- Change the term Reset to Clear- Change the term Results to Next

When graphics design the bottom bar we need to ensure that it looks clickable and static

Re-wording the Reset and Results options might also help:

We suggest Clear instead of Reset, and Next instead of Results.

Graphics team could ensure they look click-able.17


Results Screen

Users seemed interested in the videos, we recommend trying to show at least one video above the fold

If possible, have the algorithm display at least one video above the fold18

Compare designsFeatures more/less appealing?Webpage Comparisons

We also have three proposed website designs for our After Deployment website.

The homepage and topics were compared and general feedback was recorded.19

Menu button confusion

Cant identify with soldierMilitary emphasis

People in uniform more identifiable as militaryLess identifiable imagery

Soldier is standing way in the background4/72/71/7Home Page

Most users responded positively to illustrated images, which they could relate to. The distant characters and yellow colors were less appealing.20

Organized but compact

Topics seem too small and busyColors and illustrations

Images are close to homeDistracting and depressing

The pictures and colors are really negative5/72/70/7


They mostly found the illustrations more identifiable and welcoming. Yellow and orange colors were less appealing21

Card Sorting22Closed-CardPre-defined categoriesTypically takes 10-15 minutesOpen-CardUser-defined categoriesTypically takes 20-30 minutes

We also did closed- and open-card sorting with the After Deployment list of symptoms.

This was done individually, and we wanted to see if each would combine the terms into similar groups.22

One unhappy participant telling us how they feel about their current work situation.23

Card Sorting ResultsConsistent CategoriesProblems with drinking (100% agreement)Problems with people (67% agreement)24Inconsistent CategoriesProblems with feelings (35% agreement)Problems with thoughts (41% agreement)

We saw agreement for the Problems with drinking and Problems with people categories.

Feelings and Thoughts had less agreement when compared among the four users.24


Card Sorting Analysis

SPSS analysis of card sorting activity. Tree graph function combining the open- and closed-card sorting results.25


Card Sorting Analysis

Strong categories are tightly grouped on the left.

Categories which have less agreement among soldiers extend further to the right

The Social was agreed to include friends, spouse, coworkers but Thoughts and Feelings categories were not easy to distinguish for the service members.26

MentalAlways worryingRacing thoughtsAlways thinkingFeel mind doesnt work

PhysicalAlertRestlessNervousSleepNervesDistractedFeeling amped

27Card Sorting AnalysisEmotionalHopelessWorthlessAngryGuiltyAshamedNo feelingSad

The terms Mental, Physical, and Emotional were favored over Thoughts and Feelings27

Use the following words: Mental and Physical

Rather than: Thoughts and Feelings

Mental issues leading to Physical consequences28Recommendations

Difficult for users to describe the differences between feelings, and thoughts.

Rather than separate categories, theres a belief that negative thoughts lead to emotional issues, which lead to physical consequences28

Before Sorting29CATEGORYITEMSPhysical problemsPainHeadachesHead injuryProblems about feelingsSadAnxiousProblems with thoughtsRestlessEasily distractedTrouble staying focusedProblems with peopleSpouseFriends

Some of the categories currently being used29

30CATEGORYITEMSPhysical symptomsPainHeadachesHead injuryMental health issuesSadAnxiousRestlessEasily distractedTrouble staying focusedRelationshipsSpouseFriends

After Sorting

recommended changes to the category labels.

Soldiers struggled with differentiating thoughts with feelings, preferring to regard these as mental issues.30


Future TestingCategories: Tree testing symptoms and issues

Additional user testing: Tasks for experienced service members

New categories architecture could be tree tested, for the findability of issues.

And more user testing should be given to service members with deployment experience (ex: sergeants and lieutenants)31