Beyond Just Usability: Desirability & Usefulness Testing

Susan Mercer, Senior Experience Researcher @susanamercer

Usability Testing

Usability = (Efficiency + Effectiveness) In a given

context of use

Usable

Cute

Acerbic

Usability  

Usefulness  Desirability  

Usability  Consistent  

Organized  

Understandable  

Simple  

Efficient  

Usefulness  

Effec=ve  

Essen=al  

Relevant  

Helpful  

Suppor=ng  

Desirability  

Exci=ng  

Fresh  

Friendly  

Encouraging  

Meaningful  

Usability  

Usefulness  Desirability  

DESIRABILITY TESTING METHODS

Open-­‐Ended  Ques,oning • What  do  you  think  of…?

• What  is  your  reac,on  to…?

•  How  would  you  describe…?

Sentence  Comple,on •  This  app  makes  me  feel…

•  If  I  used  this  all  day,  I  would… •  Reading  on  this  website  is… •  Compared  to  other  apps,  this  app  is…

•  I  want  to  use  this  because… •  I  do  not  want  to  use  this  because…

Product  Reac,on  Cards •  118  words

•  58%  posi,ve •  36%  nega,ve •  6%  unclear

•  21%  synonyms  with  1+  other  words •  Mostly  posi,ve  words

Product  Reac,on  Cards  –  Alternate  List

•  Removed  synonyms

•  Added  opposites  to  create  Posi,ve  /  Nega,ve  Pairs •  Compared  against  two  other  emo,onal  word  vocabularies  for  Product  Appeal  and  Emo,ons

•  Marc  Hassenzhahl,  Darmstadt  University  of  Technology,  Germany

•  Geneva  Emo,on  Wheel

Annoying   Costly   Expected   Interes=ng   Poor  quality     Trivial  Apathe=c   Crea=ve   Familiar   Irrelevant   Predictable   UnaJrac=ve  AJrac=ve   Desirable   Fast   Managable   Professional   Uncontrollable  Boring   Difficult   Flexible   Meaningful     Relaxing   Understandable    Busy   Difficult   Fragile   Mo=va=ng     Relevant   Undesirable  Calm   Disappointed   Fresh   Nondescript   Reliable   Unexci=ng  Cheap   Discouaraging   Friendly   Not  Secure   Rigid   Unimagina=ve  Clean   Disorganized   Frustra=ng   Not  Valuable   Sa=sfying     Unimportant  Clear   Dull   Fun   Novel   Secure   Unnecessary  CluJered   Easy  to  use               Hard  to  Use   Obstruc=ng   Simple   Unpredictable  Common   Effec=ve   High  quality   Old   Slow   Unprofessional  Compelling   Efficient   Impersonal   Op=mis=c   Sophis=cated   Unrefined  Complex   Effortless   Impressive   Ordinary   Stable   Unreliable  Comprehensive   Empowering   Incomplete   Organized   Standard   Unusable  Confident   Encouraging   Incomprehensible   Original   Strange   Unwelcoming  Confusing   Enthusias=c   Inconsistent     Overwhelming     Stressful   Usable  Conserva=ve   Essen=al     Ineffec=ve   Patronizing   Suppor=ng   Useful  Consistent   Excep=onal   Inefficient   Personal   Tedious   Useless  Controllable         Exci=ng   Innova=ve   Pessimis=c   Time-­‐consuming   Valuable  Convenient   Exclusive   Inspiring   Pleasant   Time-­‐Saving     Weak  

AJrac=ve   UnaJrac=ve   Excep=onal   Common   Organized   Disorganized  Calm   Busy   Exci=ng   Dull   Original   Ordinary  Clean   CluJered   Exclusive   Standard   Personal   Impersonal  Clear   Confusing   Familiar   Strange   Pleasant   Annoying  Compelling   Unimportant   Fast   Slow   Predictable   Unpredictable  Comprehensive   Incomplete   Flexible   Rigid   Professional   Unprofessional  Confident   Weak   Fresh   Old   Relaxing   Stressful  Consistent   Inconsistent   Friendly   Unwelcoming   Relevant   Irrelevant  Controllable   Uncontrollable   Frustra=ng   Encouraging   Reliable   Unreliable  Convenient   Difficult   Fun   Tedious   Sa=sfying   Disappointed  Costly   Cheap   High  quality   Poor  quality   Secure   Not  Secure  Crea=ve   Unimagina=ve   Impressive   Nondescript   Simple   Complex  Desirable   Undesirable   Innova=ve   Conserva=ve   Sophis=cated   Unrefined  Easy  to  use   Hard  to  use   Inspiring   Unexci=ng   Stable   Fragile  Effec=ve   Ineffec=ve   Interes=ng   Boring   Suppor=ng   Obstruc=ng  Efficient   Inefficient   Manageable   Overwhelming   Time-­‐saving   Time-­‐consuming  Effortless   Difficult   Meaningful   Trivial   Understandable   Incomprehensible  Empowering   Patronizing   Mo=va=ng   Discouraging   Usable   Unusable  Enthusias=c   Apathe=c   Novel   Expected   Useful   Useless  Essen=al   Unnecessary   Op=mis=c   Pessimis=c   Valuable   Not  Valuable  

Reliable   Accessible   Appealing   Confusing  

Comfortable   Organized   Professional   Busy  

Confident   Straight  Forward   Approachable   Complex  

Effec=ve   Understandable   Efficient   Dull  

Friendly   Easy  to  Use   Flexible   Hard  to  Use  

Invi=ng   Clear   Helpful   Impersonal  

Time-­‐saving   Convenient   Overwhelming  

Trustworthy   Clean   Boring  

Useful   Familiar   Frustra=ng  

Simplis=c   In=mida=ng  

Valuable   Stressful  

Usable   Time-­‐consuming  

Customers  Customers  &  Non-­‐Customers   Non-­‐Customers  

68   67  

62  

42  

20  22  

17  

31  

0  

10  

20  

30  

40  

50  

60  

70  

Exis1ng   Design  A   Design  B   Design  C  

Coun

t  

Number  of  Words  

Posi=ve    

Nega=ve  

Exis1ng   Design  A   Design  B   Design  C  

Appealing   Appealing   Appealing   Appealing  

Clean   Clean   Clean   Clean  

Efficient   Efficient   Efficient   Efficient  

Engaging   Engaging   Engaging  

Intui=ve  

Professional   Professional   Professional  

Useful   Useful   Useful   Useful  

Total   9   20   16   9  

Product  Reac,on  Dyads Rate  on  a  5-­‐point  scale  from  Posi,ve  –  Nega,ve  word  pairs

AJrac=ve   ¤   ¢   ¢   ¢   ¢   UnaJrac=ve  

Calm   ¢   ¤   ¢   ¢   ¢   Busy  

Clean   ¢   ¤   ¢   ¢   ¢   CluJered  

Clear   ¤   ¢   ¢   ¢   ¢   Confusing  

Comprehensive   ¢   ¢   ¢   ¤   ¢   Incomplete  

-­‐3.5   -­‐2.5   -­‐1.5   -­‐0.5   0.5   1.5   2.5   3.5  

Discouraging  

Unrefined  

Useless  

Complex  

Irrelevant  

Boring  

Expected  

Unprofessional  

Mo=va=ng  

Sophis=cated  

Useful  

Simple  

Relevant  

Interes=ng  

Novel  

Professional  

Photo  Selec,on

LEM  Tool   (Susa  Group)

Open-­‐Ended  Ques,oning

Sentence  Comple,on

Product  Reac,on  Cards

Product  Reac,on  Dyads

Photo  Selec,on

LEM  Tool

Usability  

Desirability   Usefulness  

USEFULNESS TESTING

Pre-­‐Usability  Interview   •  How  do  you  reconcile  invoices  today…? • Who  needs  to  approve  these  and  why…?

• Where  do  you  track  which  claims  have  been  processed…?

• When  a  new  pa,ent  arrives,  what  do  you  do?

During  Usability  Tes,ng Watch  for  comments  about  usefulness

•  “Wow.  This  would  really  make  it  easier  to  reorder  the  same  things.”

•  “I  don’t  like  this.  It  would  take  me  too  long  to  do  this  part.”

Watch  for  neutral  reac,ons

•  “Eh.  This  is  OK.” •  Probe:  “How  would  this  help,  or  not  help,  you  do  your  job?”

Post-­‐Usability  Interview  

Show  screens  again  and  ask:

•  “How  would  this  work  for  you  in  your  job?” •  “What  about  this  works  for  you?  What  doesn’t  work?”

TESTING  METHODS  

Desirability  Tes,ng: •  Open-­‐Ended  Ques,oning

•  Sentence  Comple,on •  Product  Reac,on  Cards

•  Product  Reac,on  Dyads

•  LEM  Tool

Usefulness  Tes,ng: •  Pre  &  Post-­‐Interview  Ques,ons

•  Listening  &  Asking  Probing  Ques,ons

Thank You!

Susan Mercer smercer@madpow.com @susanamercer