cheap& quick internal user testing
Post on 24-May-2015
781 Views
Preview:
DESCRIPTION
TRANSCRIPT
© 2008 Handmade Mobile. All Rights Reserved. 1Mobile Design UK
3 Dec 2008
Jaeseung Chang (jaeseung@flirtomatic.com)Usability Architect @ Handmade Mobile Entertainment Ltd.
Cheap & Quick Internal User Testing
- Flirtomatic’s Recent Challenge -
© 2008 Handmade Mobile. All Rights Reserved. 2Mobile Design UK
Quick Introduction
- BA in Education from SNU
- MSc in Information Technology from QMUL
- PgDip in HCI from UCLIC
- MSc by Research in Computer Science from QMUL
- Open Education (Korea) / Usability Designer
- Samsung Electronics (Headquarter in Korea)
/ Mobile Usability Designer + Researcher
- Handmade Mobile (Flirtomatic) / Mobile Usability Architect
© 2008 Handmade Mobile. All Rights Reserved. 3Mobile Design UK
Cheap & Quick Internal User Testing
– Flirtomatic’s Recent Challenge
1. Why Cheap & Quick?
2. Experiment Design
3. Good Lessons – Dos and Donts
© 2008 Handmade Mobile. All Rights Reserved. 4Mobile Design UK
1. Why Cheap & Quick?1.1. Evaluation Needs
Requirement
Analysis
Fast
Prototyping
SpecificationImplementation
EvaluationEvaluation
- We all know the necessity of usability evaluation
- At least 2 times per 1 iteration cycle
- But the reality sometimes doesn’t help
© 2008 Handmade Mobile. All Rights Reserved. 5Mobile Design UK
1. Why Cheap & Quick?1.2. Usability Firms
- Hiring professional research firms?
- Excellent facilities, professional experiment design,
reliable recruitment, & nice report (maybe?)
- But costs a fortune!
© 2008 Handmade Mobile. All Rights Reserved. 6Mobile Design UK
1. Why Cheap & Quick?1.3. Self-Evaluation
- Then we should go DIY
- Venue: How about using our meeting room?
- Experiment design: We can do that maybe?
- Recruitment: Find some volunteers from our users?
- Observation and analysis?: 15 day trial version MORAE!
© 2008 Handmade Mobile. All Rights Reserved. 7Mobile Design UK
© 2008 Handmade Mobile. All Rights Reserved. 8Mobile Design UK
Cheap & Quick Internal User Testing
– Flirtomatic’s Recent Challenge
1. Why Cheap & Quick?
2. Experiment Design
3. Good Lessons – Dos and Donts
© 2008 Handmade Mobile. All Rights Reserved. 9Mobile Design UK
2. Experiment Design2.1. Overview
- The 1st iteration of our ‘Redesign Project’
- Based on wireframes (workflows) produced so far
- Evaluation on our first redesign UI/IA works
- To discover good/bad points for more future works
- Included three parts: UT + GUI Preference + FGD
User TestingGUI
Preference
Focus Group
Discussion+ +
© 2008 Handmade Mobile. All Rights Reserved. 10Mobile Design UK
2. Experiment Design2.2. User Testing
- Normal user testing techniques (think-aloud)
- Went through 6 sample tasks with 8 participants
- Used a hi-fi prototype to simulate our new UI
- Controlled experiment with video recording (MORAE)
- 1 moderator and multiple observers
CONFIDENTIALCONFIDENTIAL
© 2008 Handmade Mobile. All Rights Reserved. 11Mobile Design UK
2. Experiment Design2.3. GUI Preference
- Asked the same participants to choose our new GUI designs
- A sort of multiple-choice survey questions
- Asked to choose the best 2 & the worst 1 design
CONFIDENTIAL
© 2008 Handmade Mobile. All Rights Reserved. 12Mobile Design UK
2. Experiment Design2.4. Focus Group Discussion
- To collect real users’ preferences and behaviours
- In terms of using the current version of Flirtomatic
- Focused on gathering additional information
- To help redesign the details of our WAP service
- Went through 5 topics with 3 groups of users (m, f, & gay)
© 2008 Handmade Mobile. All Rights Reserved. 13Mobile Design UK
Cheap & Quick Internal User Testing
– Flirtomatic’s Recent Challenge
1. Why Cheap & Quick?
2. Experiment Design
3. Good Lessons – Dos and Donts
© 2008 Handmade Mobile. All Rights Reserved. 14Mobile Design UK
3. Good Lessons – Dos and Donts3.1. Outcomes
- Were able to conduct evaluation ourselves – DIY!
- Captured some important findings to validate our prototype
- Less than 2 wks (Experiment design + experiment + analysis)
- Less than 500.00 GBP
- Much quicker and cheaper than hiring research firms
- Were able to share the importance of usability evaluation
with the management level and all other departments
- Self-learn about the process of usability evaluation
© 2008 Handmade Mobile. All Rights Reserved. 15Mobile Design UK
3. Good Lessons – Dos and Donts3.2. Difficulties
- Recruitment error: No show up, recruited friends, etc.
- MORAE error (due to the inexperienced operation):
No video recording, low quality audio recording, etc.
- Some mistakes of our colleagues:
intruding into the UT room to connect to wireless network,
interrupting UT sessions to ask questions to the moderator,
giving money to participants during the session, etc.
© 2008 Handmade Mobile. All Rights Reserved. 16Mobile Design UK
3. Good Lessons – Dos and Donts3.3. Dos
- Good preparation in every way:
1. Documentation on task scenario, forms, analysis
2. Train yourself to master any SW tools (like MORAE)
3. Clean up the venue in advance
- Share the importance of usability evaluation with everyone
1. With the management level
2. Especially with colleagues nearby the UT room
(should politely ask silence during each session)
- Share the formal process of evaluation with team members
- Educate yourself how to run each session professionally
© 2008 Handmade Mobile. All Rights Reserved. 17Mobile Design UK
3. Good Lessons – Dos and Donts3.4. Donts
- Do not rely on the quick list of recruited users
(importance of having some back-up participants)
- Do not expect everyone else in office will behave as you do
1. They don’t know how important it is
2. Actually they don’t care what you do
- Do not expect your team members understand everything
1. They don’t know how to formally run each session
2. Importance of pre-training for usability evaluation
© 2008 Handmade Mobile. All Rights Reserved. 18Mobile Design UK
3. Good Lessons – Dos and Donts3.5. Resources
- MORAE: http://www.techsmith.com/morae.asp
- The Usability Methods Toolbox by James Hom:
http://jthom.best.vwh.net/usability
- Usability First: http://www.usabilityfirst.com/
- Jakob Nielsen: http://www.useit.com/
© 2008 Handmade Mobile. All Rights Reserved. 19Mobile Design UK
Thank you
Jaeseung Chang
(jaeseung@flirtomatic.com)
top related