wireframe testing and card sorting
Post on 16-Apr-2017
Embed Size (px)
Paper Prototype Testing Card Sorting Webpage Comparisons
User TestingWebpage Comparisons Card Sorting
After Deployment / Guided NavigationMedically Ready ForceReady Medical Force1
My job with National Center for Telehealth and Technology. On a military base, working with Service Members and their Care Providers.
After Deployment website resource for SM concerned about overseas/deployment experiences.
Guided Navigation mobile tool meant to connect SM with relevant content
Three activities: user testing with paper prototypes, card sorting of webpage categories, and qualitative feedback for three proposed webpage designs1
2After Deployment / Guided Navigation
After deployment website. Topics of issues are given in a grid, with assessments and other resources in the menu2
A vs. B
Guided navigation wireframes designed by our UX Designer.
Design A on left with menu options on top of screen with collapsible categories.
Design B has two screens. First screen is the menu options, Second with expanded categories.3
User Testing Explosive Ordnance Disposal18 ParticipantsLt. SSgt. C+16 members from 3D EOD BN
18 total participants from EOD Battalion, a very young bomb disposal squad
Point of contact: Lt. S4
User Testing5# ParticipantsTest4Wireframes A, Web Comparison5Wireframes B, Web Comparison4Closed-Card Sort5Open-Card Sort
They were divided among the three research activities:
User testing with 2 wireframes, open and closed card sorting, and the website design comparisons5
A vs. B
We wanted to see if young soldiers, with no deployment experience, would explore the Curious level of interest
We also wanted to know if the categories made sense, and if the results were expected / helpful.6
Off-site testing with prototypesUser Testing
Testing was carried out on site, at our North Fort. We were permitted to only use the paper prototypes of our design7
In the following scenario, youre a newly recruited active duty soldier.Youve heard about soldiers exercising less when returning home; youre seeking information...
A scenario was given related to their experience level.8
Feedback for design featuresDoes the layout make sense?User Testing
Feedback was recorded during this process9
User Feedback Wireframes A102/4 users skipped I am curious4/4 said the categories make sense1/4 commented on the Reset buttonReset would clear everything?
Out of four users tested with first wirefram, 2 skipped the curious option, a required first step.
When they were told, nothing seems to happen they were able to identify I am curious up top.
All four agreed with the category terms. One commented on the bottom Reset button, not sure if it would clear the check boxes or start all over10
Explore functionalityAble to find information?User Testing
User Feedback Wireframes B125/5 users selected I am curious
3/5 commented on I need helpA personal problem?Help and concerned are similarIf I were concerned, I would also need help
For wireframe Bs first screen, all five users identified I am curious
Three commented on the other options, suggesting a re-wording or combining them12
User Feedback Wireframes B135/5 said the categories make sense3/5 commented on the Reset / Results buttonsReset would start all over?Results is confusing from the past? Averages?Id make these options brighter to pop out
All five users navigated the categories successfully.
Three commented on the reset/results buttons, not sure about what theyre for, and suggested they stand out more.13
User Feedback Results Screen14
9/9 users liked the results overallGood informationI think theyre usefulResults are what was expected5/9 users liked the VideosId click on videos first, then read what theyre aboutId like to see more videos, dont like to read a lotId watch the video we get surveyed constantly
For our stretched-out results page, all nine responded positively to the results content.
Five showed a preference for videos14
Recommendations15Selecting a Level of Interest
When the user was prompted to select a level of interest (Curious, Concerned or Need Help) Design B was most successful
Based on user testing results these are the design recommendations:
Based on our findings, Design B is recommended for its level of interest screen15
Both designs were successful for category selectionsCombining the drop-down menus with color icons is recommended
Design ADesign B+=
For the categories section, Option A and B both did well;
Users responded positively to the drop-down menu and icons. A proposed combination of features is given here.
We were unable to test swiping down to search for additional categories, which could be an issue with the expanded categories of Option B.16
Recommendations17Reset / Results Bar
Users seemed confused with the terms Reset and Results- Change the term Reset to Clear- Change the term Results to Next
When graphics design the bottom bar we need to ensure that it looks clickable and static
Re-wording the Reset and Results options might also help:
We suggest Clear instead of Reset, and Next instead of Results.
Graphics team could ensure they look click-able.17
Users seemed interested in the videos, we recommend trying to show at least one video above the fold
If possible, have the algorithm display at least one video above the fold18
Compare designsFeatures more/less appealing?Webpage Comparisons
We also have three proposed website designs for our After Deployment website.
The homepage and topics were compared and general feedback was recorded.19
Menu button confusion
Cant identify with soldierMilitary emphasis
People in uniform more identifiable as militaryLess identifiable imagery
Soldier is standing way in the background4/72/71/7Home Page
Most users responded positively to illustrated images, which they could relate to. The distant characters and yellow colors were less appealing.20
Organized but compact
Topics seem too small and busyColors and illustrations
Images are close to homeDistracting and depressing
The pictures and colors are really negative5/72/70/7
They mostly found the illustrations more identifiable and welcoming. Yellow and orange colors were less appealing21
Card Sorting22Closed-CardPre-defined categoriesTypically takes 10-15 minutesOpen-CardUser-defined categoriesTypically takes 20-30 minutes
We also did closed- and open-card sorting with the After Deployment list of symptoms.
This was done individually, and we wanted to see if each would combine the terms into similar groups.22
One unhappy participant telling us how they feel about their current work situation.23
Card Sorting ResultsConsistent CategoriesProblems with drinking (100% agreement)Problems with people (67% agreement)24Inconsistent CategoriesProblems with feelings (35% agreement)Problems with thoughts (41% agreement)
We saw agreement for the Problems with drinking and Problems with people categories.
Feelings and Thoughts had less agreement when compared among the four users.24
Card Sorting Analysis
SPSS analysis of card sorting activity. Tree graph function combining the open- and closed-card sorting results.25
Card Sorting Analysis
Strong categories are tightly grouped on the left.
Categories which have less agreement among soldiers extend further to the right
The Social was agreed to include friends, spouse, coworkers but Thoughts and Feelings categories were not easy to distinguish for the service members.26
MentalAlways worryingRacing thoughtsAlways thinkingFeel mind doesnt work
27Card Sorting AnalysisEmotionalHopelessWorthlessAngryGuiltyAshamedNo feelingSad
The terms Mental, Physical, and Emotional were favored over Thoughts and Feelings27
Use the following words: Mental and Physical
Rather than: Thoughts and Feelings
Mental issues leading to Physical consequences28Recommendations
Difficult for users to describe the differences between feelings, and thoughts.
Rather than separate categories, theres a belief that negative thoughts lead to emotional issues, which lead to physical consequences28
Before Sorting29CATEGORYITEMSPhysical problemsPainHeadachesHead injuryProblems about feelingsSadAnxiousProblems with thoughtsRestlessEasily distractedTrouble staying focusedProblems with peopleSpouseFriends
Some of the categories currently being used29
30CATEGORYITEMSPhysical symptomsPainHeadachesHead injuryMental health issuesSadAnxiousRestlessEasily distractedTrouble staying focusedRelationshipsSpouseFriends
recommended changes to the category labels.
Soldiers struggled with differentiating thoughts with feelings, preferring to regard these as mental issues.30
Future TestingCategories: Tree testing symptoms and issues
Additional user testing: Tasks for experienced service members
New categories architecture could be tree tested, for the findability of issues.
And more user testing should be given to service members with deployment experience (ex: sergeants and lieutenants)31