discount mobile usability methods
DESCRIPTION
So much of the mobile experience is context and location specific. App use tends to be more immediate, reactive and transient, prompted by something that happens in the users' environment rather than by their work schedule. Traditional lab methods really don't work too well to capture these behaviors. From a physical perspective there is seldom a video-out port to capture screen images, and there is no easy way to capture gestures or button presses. From a behavioral perspective, it's often hard to "set the scene" for your app's use when your user is sitting inside a sterile office room.So how do we cheaply and quickly gather feedback on the mobile apps that we develop? What techniques can we use to balance ecological validity with solid data collection?TRANSCRIPT
DISCOUNT MOBILE USABILITY TECHNIQUES
Getting cheap, fast, reliable product feedback for mobile
devices
Chris Nodder
Chris Nodder Consulting LLC
Share!
• Issues you face• Tips and tricks• Best/worst practices
1987
1994
1999
The issues with testing mobile devices aren’t new
Problems doing mobile usability
• Physical– How to see/record what’s going on– Many device types – which to test?
• Behavioral– Triggering/capturing the important moments– Observing the interaction without changing it– Usability labs aren’t very true-to-life
• Emotional– Many features/apps are discretionary– Emotional engagement is hard to test for
Cheap, fast, reliable: pick two
• How can you get feedback to the product team quickly and cheaply, and still feel confident about it?
• Incremental research– Each piece is cheap and fast– Each piece answers specific questions that
are preventing the team from moving on– In aggregate, the observations back each
other up and provide the reliability you need
UtilityDelight
EffectivenessEfficiency
Customer Dev’tUser needs
MetricsObservation
RITE testingMetrics
Lead User studiesEthnography
Time in project cycle Location Measurement Technique
Physical issues
flic
kr/
walt
arr
rrr
Loki
on Inte
ract
ive
Litt
le S
pri
ngs
Desi
gn
Nold
us
Conto
ur
Recruiting, location issues
• Recruiting– Require at least 3 months familiarity with current device– Remind users to bring their phone & charger– Find a way to reimburse them for data/minutes used if
not on all-you-can-eat plan– Make sure their provider has reception at your location
(if lab-based)– Do they need glasses to read phone screen? (bring
them)
• Testing tips– Room without direct overhead lights (glare)– Be prepared for higher failure rates doing tasks on
mobile devices (need to reassure users)
Capturing behavior
• Low-fidelity for concept validation– Paper prototyping
• Higher fidelity for interaction validation– Flash, DHTML either on phone or on PC– Emulator studies– Competitor studies– Early builds
• On-phone (user’s phone) as soon as possible– Must be stable enough– Gather metrics– OTA updates if possible (roll out bug fixes)– Diary studies via twitter and e-mail
Emotional element (delight)
• How do we measure engagement?– Amount of use (and use over time) is a
proxy– Desirability toolkit (Product Reaction
Cards)– Analysis of adjectives used in
forums/blog postings
User experience over time
1 weekbefore
4 weeksafter
Anticipation
OrientationLearn about the
product
IncorporationUse the product in
everyday life
IdentificationDifferentiate self from
others
Ease of useStimulation
UsefulnessFits daily rituals
Social impactStimulation
What “good product” means at each phase
Karapanos et al, CHI 2009
AccessibleAdvancedAnnoyingAppealingApproachableAttractiveBoringBusiness-likeBusyCalmCleanClearCollaborativeComfortableCompatibleCompellingComplexComprehensiveConfidentConfusingConnectedConsistentControllableConvenient
CreativeCustomizableCutting edgeDatedDesirableDifficultDisconnectedDisruptiveDistractingDullEasy to useEffectiveEfficientEffortlessEmpoweringEnergeticEngagingEntertainingEnthusiasticEssentialExceptionalExcitingExpectedFamiliar
FastFlexibleFragileFreshFriendlyFrustratingFunGets in the wayHard to UseHelpfulHigh qualityImpersonalImpressiveIncomprehensibleInconsistentIneffectiveInnovativeInspiringIntegratedIntimidatingIntuitiveInvitingIrrelevantLow Maintenance
MeaningfulMotivatingNot SecureNot ValuableNovelOldOptimisticOrdinaryOrganizedOverbearingOverwhelmingPatronizingPersonalPoor qualityPowerfulPredictableProfessionalRelevantReliableResponsiveRigidSatisfyingSecureSimplistic
SlowSophisticatedStableSterileStimulatingStraight ForwardStressfulTime-consumingTime-SavingToo TechnicalTrustworthyUnapproachableUnattractiveUncontrollableUnconventionalUnderstandableUndesirableUnpredictableUnrefinedUsableUsefulValuable
www.microsoft.com/usability/uepostings/desirabilitytoolkit.doc
Extra time…
Involving the team
• List of questions team has–Write down how each will be answered–Write down answers as they come in…this way team has a stake in finding answers
• RITE testing: team must attend• Metrics: team must code into product• Field visits: encourages user empathy
RITE (Rapid Iterative Testing and Evaluation)• Ship an improved interface as rapidly and cheaply
as possible– More important to find and fix big issues than to find
every issue
• Fix issues as they are found in a study, run only enough users to ensure the fix worked– Development team must agree what users should be
able to achieve with the system (helps define issue severity)
– Development team must attend, agree issue fix, be prepared to code fixes “on the fly”
– Usability Engineer must be experienced in domain and in typical user issues to calculate level of severity
RITE - fixing issues
• Categories of issues1. Issues with obvious cause and solution, quick fix
Fix and test with next participant
2. Issues with obvious cause and solution, big fix Start fix now, test with fixed prototype when stable
3. Issues with no obvious cause (or solution) Keep collecting data, upgrade issue to 1 or 2
4. Issues caused by other factors (test script, participant) Keep collecting data, learn from mistakes
… allows you to test fixes in the same study… not an excuse for sloppy coding, UX work
RITE - Age of Empires II example
Vertical lines are revisions to test code
“Blip” = more errors seen after blocking issues removed
Extra users tested to see fixes worked
Forrester mobile app model• Handy structure for thinking about mobile user testing• The five contexts which are amplified by mobile are: location, locomotion,
immediacy, intimacy and device. – Location: People use apps in a wide variety of locations, which can be
determined through the use of GPS. – Locomotion: Mobile users access their devices while on the move - walking,
running and even (unfortunately), driving. If a phone has an accelerometer, the app can detect the motion, speed and direction of the device.
– Immediacy: Mobile users are not stationary - they need a mobile app to immediately react to find a price, transfer funds or update their status, for example. They'll be even more pleased when the app combines immediacy with location and locomotion info to anticipate their needs.
– Intimacy: Mobile users identify with their device, but designing for intimacy means you have to understand each person's relationship with their device. For example, a bargain shopper may love getting in-store coupons via push notifications, but another user may hate it.
– Device: Finally, developers should take into consideration the features specific to the device, including the varying form factors, plus the device's touch, voice recognition and image recognition capabilities.
Mobile app design best practicesMike Gualtieri, Forrester Research
