agile australia 2014
TRANSCRIPT
![Page 1: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/1.jpg)
@amir_ansari @dius_au1
UX: more than a gut feel
Amir Ansari – DiUS
![Page 2: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/2.jpg)
@amir_ansari @dius_au2
6’3”
![Page 3: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/3.jpg)
@amir_ansari @dius_au3 http://lyndabuitrago.com/tag/leaky-gut/
![Page 4: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/4.jpg)
@amir_ansari @dius_au4
![Page 5: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/5.jpg)
@amir_ansari @dius_au5
![Page 6: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/6.jpg)
@amir_ansari @dius_au6
product and technology strategy
application-specific devices
web applications
mobile applications
![Page 7: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/7.jpg)
@amir_ansari @dius_au7http://dralexkent.files.wordpress.com/2011/12/gut.jpg
We feel we know…• who our customers are• what their pain points & barriers are • what will trigger them to use our product • this product will be a good idea
![Page 8: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/8.jpg)
@amir_ansari @dius_au8
What is UX?
![Page 9: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/9.jpg)
@amir_ansari @dius_au9
http://www.uxisnotui.com/
![Page 10: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/10.jpg)
@amir_ansari @dius_au10
Study and Design of ALL aspects of a person’s EXPERIENCE with a product or service
![Page 11: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/11.jpg)
@amir_ansari @dius_au11
![Page 12: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/12.jpg)
@amir_ansari @dius_au12
![Page 13: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/13.jpg)
@amir_ansari @dius_au13
The sooner we can find which features are worth investing in, the sooner we can focus our limited resources on the best solution to our business problems.
Jeff Gothelf – LEAN UX
“
![Page 14: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/14.jpg)
@amir_ansari @dius_au14
Quantitative Qualitative
What? Why?• Data is solid and repeatable• Confirm and deduce• Result-oriented• Large sample sizes
• Data is rich and deep• Explore and describe• Process-oriented• Small sample sizes
• A/B or multi-variant testing• Analytics• Remote un-moderated
usability testing• Surveys
• Focus groups• Cognitive walkthroughs• Moderated usability
testing• Card sorting
![Page 15: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/15.jpg)
@amir_ansari @dius_au15
![Page 16: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/16.jpg)
@amir_ansari @dius_au16www.r-66.info
Completion rate
Task time
Task level satisfactionConversions
Page viewsSystem Usability Scale (SUS)
Net Promoter Score (NPS)
![Page 17: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/17.jpg)
@amir_ansari @dius_au17
Qualitative
Why?• Data is rich and deep• Explore and describe• Process-oriented• Small sample sizes
• Focus groups• Cognitive walkthroughs• Moderated usability
testing• Card sorting
![Page 18: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/18.jpg)
@amir_ansari @dius_au18
How do we measure and
validate these? And When?
![Page 19: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/19.jpg)
@amir_ansari @dius_au19
Visioning Iterations Iterations Iterations Iterations MVPLaunch
Inception & Planning
1 2 3
![Page 20: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/20.jpg)
@amir_ansari @dius_au20
Technique 1
![Page 21: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/21.jpg)
@amir_ansari @dius_au21
![Page 22: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/22.jpg)
@amir_ansari @dius_au22
Visioning Iterations Iterations Iterations Iterations MVPLaunch
Inception & Planning
1 2 3
![Page 23: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/23.jpg)
@amir_ansari @dius_au23
30 – 60 mins 3 One on one interviews
![Page 24: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/24.jpg)
@amir_ansari @dius_au24
• Concerns• Fears• Pain points• Ideal experience• Ideal channel
OUTCOMES
![Page 25: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/25.jpg)
@amir_ansari @dius_au25
• Some had not come prepared• Stakeholders wanted to ask
‘speculative questions’
![Page 26: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/26.jpg)
@amir_ansari @dius_au26
• Stakeholders observing• Call centre Staff felt engaged
and included • More insights than we had
anticipated
![Page 27: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/27.jpg)
@amir_ansari @dius_au27
Technique 2
![Page 28: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/28.jpg)
@amir_ansari @dius_au28
![Page 29: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/29.jpg)
@amir_ansari @dius_au29
Visioning Inception & Planning
1 2 3
Iterations Iterations Iterations Iterations MVPLaunch
![Page 30: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/30.jpg)
@amir_ansari @dius_au30
Focus group 3
2 hours
Focus group 1
Focus group 224
![Page 31: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/31.jpg)
@amir_ansari @dius_au31
![Page 32: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/32.jpg)
@amir_ansari @dius_au32
OutcomesFeature 1 Feature 2 Feature 3
![Page 33: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/33.jpg)
@amir_ansari @dius_au33
• 2 hours wasn’t quite enough• Running sessions back to back
in the same evening!
![Page 34: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/34.jpg)
@amir_ansari @dius_au34
• Stakeholders buy-in through observation• Validate assumptions• Client was anonymous
![Page 35: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/35.jpg)
@amir_ansari @dius_au35
Technique 3
![Page 36: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/36.jpg)
@amir_ansari @dius_au36
Visioning Inception & Planning
1 2 3
Iterations Iterations Iterations Iterations MVPLaunch
![Page 37: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/37.jpg)
@amir_ansari @dius_au37
1 hour One on one testing6
![Page 38: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/38.jpg)
@amir_ansari @dius_au38
http://www.nngroup.com/articles/how-many-test-users/
![Page 39: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/39.jpg)
@amir_ansari @dius_au39
![Page 40: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/40.jpg)
@amir_ansari @dius_au40
![Page 41: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/41.jpg)
@amir_ansari @dius_au41
• Refinements sometimes not subtle enough• Design ping pong
![Page 42: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/42.jpg)
@amir_ansari @dius_au42
• Speed of testing• The feedback loop• Stakeholder buy-in
(live stream)• Rapid de-brief• One designs and one tests
![Page 43: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/43.jpg)
@amir_ansari @dius_au43
1. Background
2. Falsifiable Hypotheses
3. Details
6. Next Actions
5. Validated Learning
4. Results
User 1 User 2 User 3 etc.
Hypothesis 1 Hypothesis 2 Etc.
• What feature or assumption to test• What questions to ask• How well does X perform• Etc. • What to test next
• What hypothesis do we want to validate
1. That majority of users think X2. That majority of users perform Y 3. That majority of users need Z4. Etc.
• Previous experiment taught us …• We want to find out…• We want to test our solutions…
1. We can proceed with X2. We need to tweak Y3. We need to pivot and get rid of Z
Title: Experiment 2: XXX Author: Created:
• From focus groups, it was clear that one barrier with other online providers has been the lack of trainer support
• That providing access to trainers in the classroom and telling students upfront about trainer availability will eliminate barriers and increase conversion
• Show users the marketing pages that talk about trainer availability
• Show users the classroom with the trainer being online
• Users indeed look for trainer availability and are comforted by knowing OTI provides access to them
• Test something else! ; )
![Page 44: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/44.jpg)
@amir_ansari @dius_au44
bit.ly/rapid-debrief
![Page 45: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/45.jpg)
@amir_ansari @dius_au45
![Page 46: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/46.jpg)
@amir_ansari @dius_au46
![Page 47: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/47.jpg)
@amir_ansari @dius_au47
![Page 48: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/48.jpg)
@amir_ansari @dius_au48
![Page 49: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/49.jpg)
@amir_ansari @dius_au49
![Page 50: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/50.jpg)
@amir_ansari @dius_au50
5
One on one testingFocus groups
= $8002 FTEs
X $160X $375*
* Daily gross equivalent based on a gross annual salary of $90k
= $750
$1,550
16 = $3,2003 FTEs
X $200X $375*= $1,125
$4,325
If doing this internally…
![Page 51: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/51.jpg)
@amir_ansari @dius_au51
![Page 52: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/52.jpg)
@amir_ansari @dius_au52
Define why, what, when who, where how, how often
1
![Page 53: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/53.jpg)
@amir_ansari @dius_au53
Become friends with the PM (or the person with the budget)
2
![Page 54: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/54.jpg)
@amir_ansari @dius_au54
Keep it lean and measure often
3
![Page 55: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/55.jpg)
@amir_ansari @dius_au55
Get stakeholder buy-in(Get them to observe)
4
![Page 56: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/56.jpg)
@amir_ansari @dius_au56
Be consistentin how you measure and capture measurements
5
![Page 57: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/57.jpg)
@amir_ansari @dius_au57
Define when to Quant and when to Qual
6
![Page 58: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/58.jpg)
@amir_ansari @dius_au58http://dralexkent.files.wordpress.com/2011/12/gut.jpg
Replace with more powerful point• who our customers are• what their pain points & barriers are • what will trigger them to use our product • this product will be a good idea
![Page 59: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/59.jpg)
@amir_ansari @dius_au59
Thank you
Qualitative data can in fact be converted into quantitative measures even if it doesn't come from an experiment or from a large sample size.
Jeff Sauro – Measuring Usability“
![Page 60: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/60.jpg)
@amir_ansari @dius_au60
Resources
![Page 61: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/61.jpg)
@amir_ansari @dius_au61http://stamfordinteractive.com.au/conversations/benefits-of-guerrilla-testing/
![Page 62: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/62.jpg)
@amir_ansari @dius_au62
You can quantify qualitative data!
bit.ly/quantifying-qualitative bit.ly/t-distribution
![Page 63: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/63.jpg)
@amir_ansari @dius_au63
Completion rate
Task timeTask level satisfaction
Conversions
Page views
System Usability Scale (SUS)
Net Promoter Score (NPS)
{Visioning Inception & Planning Iterations Iterations Iterations Iterations MVP
Launch
![Page 64: Agile Australia 2014](https://reader035.vdocuments.net/reader035/viewer/2022062904/58898cb31a28ab330e8b47c5/html5/thumbnails/64.jpg)
@amir_ansari @dius_au64
https://www.measuringusability.com/topics/UX