designing for wearables
TRANSCRIPT
DESIGNING FOR WEARABLES
DESIGNING FOR WEARABLES Mark Billinghurst [email protected]
Oct 17th 2015
AWE Asia 2015
DESIGNING FOR WEARABLES
Mark Billinghurst ▪ Professor of HCI, University of
South Australia
▪ PhD Univ. Washington
▪ Research on AR, mobile HCI, Collaborative Interfaces
▪ More than 300 papers in AR, VR, interface design, wearable
▪ Sabbatical in Glass team at Google [x] in 2013
DESIGNING FOR WEARABLES
Goals and Schedule Goals
Learn simple interface design guidelines Learn useful prototyping tools Learn where further resources are
Schedule 4:15 Introduction (5 minutes) 4:20 Design Guidelines (15 minutes) 4:35 Prototyping Tools (20 minutes) 4:55 Resources (5 minutes)
DESIGNING FOR WEARABLES
1. INTRODUCTION
DESIGNING FOR WEARABLES
DESIGNING FOR WEARABLES
Room Desk Lap Hand Head
DESIGNING FOR WEARABLES
DESIGNING FOR WEARABLES
What is a Wearable Computer? ▪ Computer on the body that is: ▪ Always on ▪ Always accessible ▪ Always connected
▪ Other attributes ▪ Augmenting user actions ▪ Aware of user and surroundings
Starner, T. E. (1999). Wearable computing and contextual awareness (Doctoral dissertation, Massachusetts Institute of Technology).
DESIGNING FOR WEARABLES
Evolution of Wearables • asdfad
DESIGNING FOR WEARABLES
Second Gen. Systems • Recon (2010 - )
• Recon Jet, Snow • Head worn displays for sports • Acquired by Intel (2015)
• Google (2011 - ) • Google Glass • Consumer focus
DESIGNING FOR WEARABLES
Smart Glass in 2015
DESIGNING FOR WEARABLES
How do you Design for this?
DESIGNING FOR WEARABLES
2. DESIGN GUIDELINES
DESIGNING FOR WEARABLES
Universal Design Principles
Flexibility
Equitable use Easy to perceive Simple and intuitive
Low physical effort
High tolerance for error
DESIGNING FOR WEARABLES
Designing for Wearables
• Wearables are intimate on-body devices, so interface design for wearables, means: • Designing for Attention
• Designing for Interruption
• Designing for User Experience
• Designing for Social Interaction
DESIGNING FOR WEARABLES
Attention
DESIGNING FOR WEARABLES
Micro-Interactions
Using mobile phone people split their attention between the display and the real world
DESIGNING FOR WEARABLES
Time Looking at Screen
Oulasvirta, A. (2005). The fragmentation of attention in mobile interaction, and what to do with it. interactions, 12(6), 16-18.
DESIGNING FOR WEARABLES
Using Micro Interactions
Quick micro-interactions reduce divided attention and allow people to spend more time in real world
DESIGNING FOR WEARABLES
It's like a rear view mirror
Don't overload the user. S2ck to the absolutely essen2al, avoid long
interac2ons. Be explicit.
DESIGNING FOR WEARABLES
Make it glanceable Seek to rigorously reduce information density. Successful designs afford for recognition, not reading.
Bad Good
DESIGNING FOR WEARABLES
Reduce the number of info chunks You are designing for recognition, not reading. Reducing the total # of information chunks will greatly increase the glanceability of your design.
1
2
3
1
2
3
4
5 (6)
Test done by Morten Just using a watch
DESIGNING FOR WEARABLES
Design Single Interactions < 4 sec
Eye movements For 1: 1 230ms For 2: 1 230ms For 3: 1 230ms For 4: 3 690ms For 5: 2 460ms
~1,840ms
Eye movements For 1: 1-2 460ms For 2: 1 230ms For 3: 1 230ms
~920ms
1
2
3
1
2
3
4
5 (6)
Test done by Morten Just using a watch
DESIGNING FOR WEARABLES
Test the glanceability of your design
DESIGNING FOR WEARABLES
Design for MicroInteractions
▪ Design interactions less than a few seconds • Tiny bursts of interaction • One task per interaction • One input per interaction
▪ Benefits • Use limited input • Minimize interruptions • Reduce attention fragmentation
DESIGNING FOR WEARABLES
Rule of Thumb: Design for limited attention/
microinteractions - no more than 4 seconds to complete a given step in the interaction
DESIGNING FOR WEARABLES
Interruption
DESIGNING FOR WEARABLES
Designing for Interruptions
• Assume user is engaged in critical real world task • Use context to filter interruptions (is it necessary?) • Interrupt in way that consumes least attention • Allow user to dismiss interruption with minimal effort • Progressively disclose information and increase interaction
DESIGNING FOR WEARABLES
Example: Interruptions on Glass
▪ Gradually increase engagement and attention load ▪ Respond to user engagement
Receiving SMS on Glass
“Bing”
Tap Swipe
Glass
Show Message Start Reply
User Look Up
Say Reply
DESIGNING FOR WEARABLES
Rule of Thumb: Design for interruption and low
cognitive load that can be increased as needed.
DESIGNING FOR WEARABLES
User Experience
DESIGNING FOR WEARABLES
Consider Your User
• Wearable User • Probably Mobile • One/no hand interaction • Short application use • Need to be able to multitask • Use in outdoor or indoor environment • Want to enhance interaction with real world
DESIGNING FOR WEARABLES
How would you take a note?
DESIGNING FOR WEARABLES
Example: Glass Pictures
• On Glass there are three ways to take a picture 1/ Voice commands – “Ok Glass, Take a Picture”
2/ Touch navigation through menu
3/ Winking with right eye
• Which you use depends on context • Riding a bike outdoors – voice commands • During a meeting – winking
DESIGNING FOR WEARABLES
Rule of Thumb: Provide multiple ways of accessing
functionality.
DESIGNING FOR WEARABLES
Design For Device
• Simple, relevant information • Complement existing devices
DESIGNING FOR WEARABLES
Glass User Interface
• dfasdf
DESIGNING FOR WEARABLES
DESIGNING FOR WEARABLES
Do one thing at a time
DESIGNING FOR WEARABLES
Test your design indoors + outdoors
DESIGNING FOR WEARABLES
Design for Context
DESIGNING FOR WEARABLES
Design for Ecosystem of Wearables
User have multiple devices - phone, watch - fitness band, HMD
Each device should be used when it’s most relevant and when it’s the easiest interaction available.
DESIGNING FOR WEARABLES
Interface Guidelines
▪ Design for device • Use multiple input options • Do one thing at a time • Consider user context ▪ Design for indoor and outdoor use ▪ Design for device ecosystem
DESIGNING FOR WEARABLES
Social Interaction
DESIGNING FOR WEARABLES
Social Acceptance
• People don’t want to look silly • Only 12% of 4,600 adults would be willing to wear AR glasses • 20% of mobile AR browser users experience social issues
• Acceptance more due to Social than Technical issues • Needs further study (ethnographic, field tests, longitudinal)
DESIGNING FOR WEARABLES
Rule of Thumb: Fashion First - It DOES NOT MATTER what
the device does unless the user is willing to put it on the first time
DESIGNING FOR WEARABLES
DESIGNING FOR WEARABLES
DESIGNING FOR WEARABLES
Rule of Thumb: Make the interface not just for the user
but also for the people around the user, both physically and socially.
DESIGNING FOR WEARABLES
Attention: least visual-manual attention necessary, < 2 second access time Interruption: assume busy, ramped attention Wearable: long term, unobtrusive, fashionable
User Experience: design for device, multiple ways of information access, all environments Social: graceful interfaces, design for others
Summary
DESIGNING FOR WEARABLES
3. PROTOTYPING
DESIGNING FOR WEARABLES
How can we quickly prototype Wearable
experiences with little or no coding?
DESIGNING FOR WEARABLES
Why Prototype?
▪ Quick visual design ▪ Capture key interactions ▪ Focus on user experience ▪ Communicate design ideas ▪ “Learn by doing/experiencing”
DESIGNING FOR WEARABLES
Typical Development Steps
▪ Sketching ▪ Storyboards ▪ UI Mockups ▪ Interaction Flows ▪ Video Prototypes ▪ Interactive Prototypes ▪ Final Native Application
Increased Fidelity & Interactivity
Static Low fidelity
Interactive High fidelity
DESIGNING FOR WEARABLES
Sketched Interfaces
▪ Sketch + Powerpoint/Photoshop/Illustrator
DESIGNING FOR WEARABLES
Paper Prototype
• Use sketched interface in template
DESIGNING FOR WEARABLES
Glass UI Templates
▪ Google Glass Photoshop Templates ▪ http://glass-ui.com/ ▪ http://dsky9.com/glassfaq/the-google-glass-psd-template/
DESIGNING FOR WEARABLES
Smart Watch Templates
• Eg https://dribbble.com/jaysuthar/buckets/260235-watch
DESIGNING FOR WEARABLES
Application Flow
DESIGNING FOR WEARABLES
Glassware Flow Designer
• Quick flow layout tool • https://glassware-flow-designer.appspot.com/
DESIGNING FOR WEARABLES
Static Limitations ▪ Positives ▪ Good for documenting screens ▪ Can show application flow
▪ Negatives ▪ No interactivity/transitions ▪ Can’t be used for testing ▪ Can’t deploy on wearable ▪ Can be time consuming to create
DESIGNING FOR WEARABLES
▪ Series of still photos in a movie format. ▪ Demonstrates the experience of the product ▪ Show transitions between states ▪ Discover where concept needs fleshing out. ▪ Communicate experience and interface ▪ You can use whatever tools, from Flash to iMovie.
Video Sketching
DESIGNING FOR WEARABLES
UI Concept Movies
DESIGNING FOR WEARABLES
DESIGNING FOR WEARABLES
Interactive Wireframing ▪ Developing interactive interfaces/wireframes ▪ Transitions, user feedback, interface design
▪ Web based tools ▪ UXpin - http://www.uxpin.com/ ▪ proto.io - http://www.proto.io/
▪ Native tools ▪ Justinmind - http://www.justinmind.com/ ▪ Axure - http://www.axure.com/
DESIGNING FOR WEARABLES
Proto.io - http://www.proto.io/ ▪ Web based mobile prototyping tool ▪ Features ▪ Prototype for multiple devices ▪ Gesture input, touch events, animations ▪ Share with collaborators ▪ Test on device
DESIGNING FOR WEARABLES
Proto.io Demo Video
DESIGNING FOR WEARABLES
Proto.io Android Wear Demo
• https://proto.io/showcase/android-wear/
DESIGNING FOR WEARABLES
Wireframe Limitations ▪ Can’t deploy on Device ▪ No access to sensor data ▪ Camera, orientation sensor
▪ No multimedia playback ▪ Audio, video
▪ Simple transitions ▪ No conditional logic
▪ No networking
DESIGNING FOR WEARABLES
Processing ▪ Programming tool for Artists/Designers ▪ http://processing.org ▪ Easy to code, Free, Open source, Java based ▪ 2D, 3D, audio/video support ▪ Sketching with code
▪ Processing For Android ▪ http://wiki.processing.org/w/Android ▪ Strong Android support ▪ Generates Android .apk file
DESIGNING FOR WEARABLES
http://processing.org/
DESIGNING FOR WEARABLES
Basic Processing Sketch /* Notes comment */ //set up global variables float moveX = 50; //Initialize the Sketch void setup (){ } //draw every frame void draw(){ }
DESIGNING FOR WEARABLES
Processing and Wearables ▪ One of the easiest ways to build rich
interactive wearable applications ▪ focus on interactivity, not coding
▪ Collects all sensor input ▪ camera, accelerometer, touch
▪ Can build native Android .apk files ▪ Side load onto Glass
DESIGNING FOR WEARABLES
Hello World Image PImage img; // Create an image variable void setup() { size(640, 360); //load the ok glass home screen image img = loadImage("okGlass.jpg"); // Load the image } void draw() { // Displays the image at point (0,0) image(img, 0, 0); }
DESIGNING FOR WEARABLES
Demo
DESIGNING FOR WEARABLES
Touch Pad Input ▪ Tap recognized as DPAD input
void keyPressed() { if (key == CODED){ if (keyCode == DPAD) {
// Do something ..
▪ Java code to capture rich motion events ▪ import android.view.MotionEvent;
DESIGNING FOR WEARABLES
Motion Event //Glass Touch Events - reads from touch pad public boolean dispatchGenericMotionEvent(MotionEvent event) { float x = event.getX(); // get x/y coords float y = event.getY(); int action = event.getActionMasked(); // get code for action switch (action) { // let us know which action code shows up case MotionEvent.ACTION_DOWN: touchEvent = "DOWN"; fingerTouch = 1; break; case MotionEvent.ACTION_MOVE: touchEvent = "MOVE"; xpos = myScreenWidth-x*touchPadScaleX; ypos = y*touchPadScaleY; break;
DESIGNING FOR WEARABLES
Demo
DESIGNING FOR WEARABLES
Sensors ▪ Ketai Library for Processing ▪ https://code.google.com/p/ketai/
▪ Support all phone sensors ▪ GPS, Compass, Light, Camera, etc
▪ Include Ketai Library ▪ import ketai.sensors.*; ▪ KetaiSensor sensor;
DESIGNING FOR WEARABLES
Using Sensors ▪ Setup in Setup( ) function
▪ sensor = new KetaiSensor(this); ▪ sensor.start(); ▪ sensor.list();
▪ Event based sensor reading void onAccelerometerEvent(…) { accelerometer.set(x, y, z); }
DESIGNING FOR WEARABLES
Sensor Demo
DESIGNING FOR WEARABLES
Physical Input Devices ▪ Can we develop unobtrusive input devices ? ▪ Reduce need for speech, touch pad input ▪ Socially more acceptable
▪ Examples ▪ Ring, ▪ pendant, ▪ bracelet, ▪ gloves, etc
DESIGNING FOR WEARABLES
Prototyping Platform
Arduino Kit Bluetooth Shield Google Glass
DESIGNING FOR WEARABLES
Example: Glove Input
▪ Buttons on fingertips ▪ Map touches to commands
DESIGNING FOR WEARABLES
Example: Ring Input
▪ Touch strip, button, accelerometer ▪ Tap, swipe, flick actions
DESIGNING FOR WEARABLES
How it works
Bracelet
Armband
Gloves
1,2,3,4
Values/output
DESIGNING FOR WEARABLES
Summary • Prototyping for wearables is similar to mobiles
• Tools for UI design, storyboarding, wireframing
• Android tools to create interactive prototypes • Processing, WearScript, etc
• Arduino can be used for hardware prototypes • Once prototyped Native Apps can be built
• Android + SDK for each platform
DESIGNING FOR WEARABLES
4. Resources
DESIGNING FOR WEARABLES
Designing For Glass Video
• https://www.youtube.com/watch?v=6ERgbIJ6pCM
DESIGNING FOR WEARABLES
Glass UI Design Guidelines
• More guidelines – https://developers.google.com/glass/design/index
DESIGNING FOR WEARABLES
GlassSim – http://glasssim.com/
▪ Simulate the view through Google Glass ▪ Multiple card templates
DESIGNING FOR WEARABLES
GlassSim Card Builder ▪ Use HTML for card details ▪ Multiple templates ▪ Change background ▪ Own image ▪ Camera view
DESIGNING FOR WEARABLES
GlassSim Samples
DESIGNING FOR WEARABLES
Pop - https://popapp.in/
• Take pictures of sketches • Link pictures together • Combining sketching and interactivity on mobiles
DESIGNING FOR WEARABLES
Using Pop
DESIGNING FOR WEARABLES
Viewing Design on Device • Android Design Preview
• https://github.com/romannurik/AndroidDesignPreview
• View a portion of your desktop on Android device • Select region of screen • Mirror it on Android Device
• Use to view mock-ups on target device • Eg Powerpoint for wearable mockups
DESIGNING FOR WEARABLES
Other Tools ▪ Wireframing ▪ pidoco ▪ FluidUI
▪ Rapid Development ▪ Phone Gap ▪ AppMachine
▪ Interactive ▪ App Inventor ▪ WearScript
DESIGNING FOR WEARABLES
WearScript • WearScript
• Combines power of Android development on Glass with the learning curve of a website
• Use a cloud IDE to write JavaScript that runs instantly on Glass inside a WebView
• Get started at http://wearscript.com
• Support for advanced features • Augmented Reality • Eye tracking • Arduino input
DESIGNING FOR WEARABLES
How it Works
DESIGNING FOR WEARABLES
Books
▪ Programming Google Glass ▪ Eric Redmond ▪ Rapid Android Development: Build Rich, Sensor-Based Applications with Processing ▪ Daniel Sauter
DESIGNING FOR WEARABLES
• Beginning Google Glass Development by Jeff Tang
DESIGNING FOR WEARABLES
• Microinteractions: Designing with Details • Dan Saffer • http://microinteractions.com/
DESIGNING FOR WEARABLES
CHI Wearables Exhibit
Online at http://wcc.gatech.edu/exhibition
DESIGNING FOR WEARABLES
Glass Resources ▪ Main Developer Website ▪ https://developers.google.com/glass/
▪ Glass Apps Developer Site ▪ http://glass-apps.org/glass-developer
▪ Google Design Guidelines Site ▪ https://developers.google.com/glass/design/
index?utm_source=tuicool ▪ Google Glass Emulator ▪ http://glass-apps.org/google-glass-emulator
DESIGNING FOR WEARABLES
Recon Jet • Developer Website
• http://www.reconinstruments.com/developers/getting-started/reconsdk/
• Application Prototyping • http://www.reconinstruments.com/developers/resources/
application-prototyping/
DESIGNING FOR WEARABLES
Other Resources ▪ AR for Glass Website ▪ http://www.arforglass.org/
▪ Vandrico Database of wearable devices ▪ http://vandrico.com/database
DESIGNING FOR WEARABLES
http://openprocessing.org/
DESIGNING FOR WEARABLES
http://toxiclibs.org/
DESIGNING FOR WEARABLES
Books ▪ Programming Google Glass ▪ Eric Redmond
▪ Rapid Android Development: Build Rich, Sensor-Based Applications with Processing ▪ Daniel Sauter