tat dynamic uis 250609
DESCRIPTION
TAT Webinar on Dynamic UIsTRANSCRIPT
James Haliburton, Concept Lead
Dynamic UIs
CONTENTS
•Today & Tomorrow’s Mobile
•Future Visions
•Emerging Tech
•Dynamic UIs• Legacy of Mobile• What’s in a phone today• Patterns for Achieving Dynamic UIs• Transitions are the new state• Design tools & processes
•Inspirational Seminars
What is a Dynamic UI?
A dynamic UI experience is about valuing the in-the-moment use of the mobile phone, and not only the product’s overall usefulness. A dynamic UI creates a pleasurable experience.
What is a Dynamic UI?
4 Principles•Provide apt feedback•Invite playful, non-task based behavior•Respond to “incorrect” interaction•Design forms of interaction which support natural behavior
Philosophy about the future
At TAT we believe that it is necessary to examine and innovate the future of technology. We are examining strong use cases which justify and direct the need for this development. We are designing for these scenarios now, as the
future is arriving quickly.
TODAY’S MOBILE
TODAY’S MOBILE
•Capacitive touchscreens•Multitouch•GPS•Bluetooth•Wifi
•Accelerometers•Compasses•High res•More processing power than it took to put a man on the moon!
TOMORROW’S MOBILE
Augmented reality
projected UI:s
GyrosGestures3D
Augmented reality
projected UI:s
GyrosGestures3D
UI/UX in mobile moving forward
Changed Input MethodsWe have recently moved from phones like the Sony Ericsson T610 with indirect input (keys) and a color graphical output to the time of the iPhone. Changes in the UI are partially enabled by the change from indirect input to direct input (touch).
But do our designs actually take into account that we have constructed new types of devices?
FUTURE VISIONS
FUTURE VISIONS
FUTURE VISIONS
EMERGING TECH
PROJECTOR PHONES
PROJECTOR PHONES: FUTURE
PROJECTOR PHONES
A Radical Shift•Not merely a matter of scaling up existing UIs and mobile content
•Offers a paradigm shift in the way we interact
•Phones have a social presence in our lives more than other technology
•They are not fixed to a location or specific task - they aim to be flexible and align to our behavior and to enhance our experiences
•Our behavior and experiences are more often than not, situated amongst other people
•Private device in a public setting causes a disparity and disconnect
•The key difference with procam enabled UIs will be that we can begin to truly interact, work, and play together. We will not be forced to remove ourselves from a conversation to check a mapping application or record a phone number - these can be collaborative actions.
•The UI can become a conversational tool.
FLEXIBLE DISPLAYS
18
A multimodal user interface is a UI that involves more than one modality, or human sense
A multimodal UI is thus a UI that provides feedback to several human senses, e.g. visual, auditory and haptic (tactile) feedback
MULTIMODAL
19
Sounds can be added to augment the visual design, adding a new dimension to user immersion
Compared to haptics, sounds can convey information efficiently, e.g. through sound effects, music or even speech
Globe Sejf Trum
MULTIMODAL
20
Sound can help users to better understand the effects of their interactions, especially when using touchscreens
Visual design themes can have corresponding sound themes, enhancing the metaphor of the GUI
Water Fire Paper
MULTIMODAL
21
As proven by film and computer games, sounds can be very expressive
Sounds should be subtle and varied, to avoid annoying people around the person using the device
Example: Asia cute UI with sound
MULTIMODAL
22
MULTIMODAL: FUTURE
23
A UI ECOSYSTEM
Mobile UIs are no longer just about button input and visual outputNew input paradigms require new approaches to output, for example:
Touchscreen input -> usability need for haptic and/or sound feedbackGesture input (via touchscreen or motion sensor) -> expectation of visual output that
follow gestures realistically, e.g. through physics modeling3D aesthetics and realism -> increasing expectations that UIs should mimic the real
world, e.g. by including sound and haptics
In a well designed device, output is closely linked and matched to user inputChanging one part of a UI will affect other aspects
MULTIMODAL
AUGMENTED REALITY
What is a gesture?•Essentially any movement can be a gesture•Rich and subtle•Occurs over space and time•Until recently we’ve been limited mostly to 1-dimensional button pressing•But touch brings in new possibilities
GESTURES
A model for understanding gestures•Symbolic•Iconic•Metaphoric
Symbolic•3x3 Grid•Must learn what the gesture means•“automagical”•Gesture is symbolic of the action occuring
GESTURES
A model for understanding gestures•Symbolic•Iconic•Metaphoric
Iconic•Uses “real world” gestures matched to actions on the wcreen•Flipping like you do your CD collection•Motion matches something we already know how to do
GESTURES
A model for understanding gestures•Symbolic•Iconic•Metaphoric
Metaphoric•Uses a familiar metaphor•Doesn’t necessarily match real world interaction•Needs support of sound, and visuals
GESTURES
A model for understanding gestures
GESTURES
SYMBOLIC ICONIC METAPHORIC
•Pro-user•magic quality•good for supporting surprise•can be seperate from the graphic UI•‘But supported by feedback in other UI elements•Can be difficult to remember•Gesture is its own layer
•Instantly understood•slower to perform than arbitrary gestures•context dependent•highly satisfying•can find inspiration in industrial design affordances•gesture is the meaning
•easily remembered•dependent on other UI elements•can be used to discover new functionality•gesture is enabler
3D & PHYSICS
3D & PHYSICS
3D & PHYSICS
Flexible Information Visualization (FIV) Describes the elements of 3D which allow a UI to better display large amounts of data, like playlists, in interesting ways relevant to the user and to the application being used.
Naturalized Interaction (NI) Describes the elements of a 3D UI which minimize how much a user needs to think about how theyʼre using an application, and allows them to simply use it.
Visual Style and Feedback (VSF) Describes how 3D can add a “wow effect” to a product and better differentiate itself from competitors.
Declutter
Optimize
Focus
Zoom & OverviewFIVRealtionships
Reduce Cognitive Load
Flatten Learning Curve
Show Options
Show AffordancesNIWOW - Effect
Visual ImmersionFeedback
Holistic & Seamless
VSF
3D & PHYSICS
3D & PHYSICS
QUALITIES FROM THIS NEW TECHNOLOGY
Integrating our devices with the real world•We can use our devices as social tools•They can be flexible to our needs•Encourage fun and playfulness•Designs can be rich and creative
TODAY’S MOBILE
•Capacitive touchscreens•Multitouch•GPS•Bluetooth•Wifi
•Accelerometers•Compasses•High res•More processing power than it took to put a man on the moon!
WHAT’S THE RESULT OF ALL THIS TECH?
A nicer box with a longer feature list.Minimal UI innovation.
WHAT’S THE RESULT OF ALL THIS TECH?
What about the UI & UX?
UIs are still based around static screen states.
This is legacy of WIMP (windows, icons, menus, pointing device), but we’ve been talking about post-WIMP for 15 years.
This makes sense with simple task based operations. Screen states support consistency and simple flows.
WHAT’S THE RESULT OF ALL THIS TECH?
PC Mobile Evolution So Far
Windows Static screens
Icons Icons, Widgets
Menus Menus in soft keys
Pointing Device (mouse, trackpad)
D-Pad, Trackball, Touch pointing
WHAT’S THE RESULT OF ALL THIS TECH?
Input innovation has been big (qwerty, touch, accelerometer, predictive text)
•But output and presentation have been mostly "cosmetics" (if that)
•iPhone did a big jump in creating a more natural interface (Realtime, direct manipulation)
•There are more steps in making devices less like machines - interface with real world feel (dynamic)
DYNAMIC UIs - Reacts to real world behaviour
DYNAMIC UIs - Persistency of elements
DYNAMIC UIs - Soft Values, Generative visuals
DYNAMIC UIs - Movable light sources
DYNAMIC UIs - Adaptive Elements
DYNAMIC UIs - Situated Change
Sony Ericsson S500i
Theme changes appearance
depending on time of day, date and special events.
Appearance changes based on the time of day and special events.
DYNAMIC UIs - Personalized Space
DYNAMIC UIs - Personalized Behavior
50
Water Fire Paper
DYNAMIC UIs - Multimodal Design
Dynamic sound can help user to better understand the effects of their interactions, especially when using touchscreens.
DYNAMIC UIs - Analogue Manipulation, Physics, Transition Free
DYNAMIC UIs
TRANSITIONS ARE THE NEW STATE
Dynamic UIs support real behaviour.
This makes sense when you use technology that connects your device to the real world.
UI Paradigm Transition Type User Reaction
Static Screen No Transition Where am I?
Static Screen Full screen transition Something happened, I’m somewhere else
Static Screen Smart full screen transition I understand where I’ve arrived
Dynamic UI Link flow transition I can do some new things now
Dynamic UI Transformation of UI elements I can repurpose these UI elements for what I want
Dynamic UI User controlled space I’m casually browsing, but have control to do what I like
DYNAMIC UIs
TRANSITIONS ARE THE NEW STATE
Dynamic UIs support real behaviour.
This makes sense when you use technology that connects your device to the real world.
DYNAMIC UIs
Dynamic UI Qualities Static State Based UI Qualities
Adaptive Fixed
User expectations are different PC based expectation
Metaphoric, natural behavior Abstract behavior
Empathetic Unforgiving
Creates dialogue with user Directing
Explorative Functional
Playful Task based
Holistic Fragmented
Can communicate soft values Hard values
DYNAMIC UIs
Dynamic UI Patterns
Persistency of UI Elements
Generative Visuals
Apt Reactions to Physical Input
Movable Light Sources
Contextual Change
Personalization of Space
Adaptive UI Elements
Synchronized Multimodal Design
Analogue Manipulation Patterns
Free Moving Elements With Physics Models
DYNAMIC UIs
Design tools & processes
Clear mental / behavioural models•The device has a persona•Range of behaviour and personality is the consistency which pulls a dynamic UI together - aligns to situations•Can derive both design patterns & tech requirements
Agile Work Process•All stakeholders involved•Horizontal, not vertical process•Rapid iterations•Important to sketch on device!
Concept Sketching
v2v2
v1
•wireframes•functional specifications•paper prototypes•flash movies•UI flows•etc.
Typical design production and documentation has difficulty communicating the qualities of a dynamic user experience.
The tools we use for sketching also leave us short as designers in actually testing the experience of what the UI will be like.
We should be able to experience and communicate the rich behavior and interactions of our UIs.
Spatial metaphors can be used to reduce cognitive load since you can work with familiar principles.
Using a metaphor can make it easier to achieve consistency throughout the UI.
It is the first step in creating a consistent behavioral model.
Communicating 3D or behvioral UIs within teams can be troublesome, to have a spatial metaphor can make discussions more direct.
A shape alone is not a metaphor.
Sketching of 3D UIs using spatial metaphors
However, don’t go overboard! It’s still a UI and you can break the rules!
Communicating camera angles in 3D UIs
Concept Sketching
Concept Sketching
Shape vs. Metaphor
•Metaphors have behaviour
•You can put your shape in an environment
•Simple is good
•Spatial metaphors are all about being consistent
•Like personas for interaction behaviour
•You are testing how the UI behaves and feels
Concept Sketching
Traditional work process
Planned
Traditional work process
Actual
Iterative work process with Cascades
DYNAMIC UIs
Patterns & Tools
3D, Touch, GesturesPhysicsGenerative movement & animationMultimodal designClear mental models, Agile work processes
A dynamic UI experience is about valuing the in-the-moment use of the mobile phone, and not only the product’s overall usefulness. A dynamic UI creates a pleasurable experience.
DYNAMIC UIs
There is no best practice.
Our UI design patterns will come from practices of physical computing, industrial design, and good field research - not the desktop PC.
Patterns & Tools
3D, Touch, GesturesPhysicsGenerative movement & animationMultimodal designClear mental models, Agile work processes
Inspirational Seminars
•3D•Android•Multimodal•Advanced Displays•Physics•Sketching & Prototyping
Creating Real Customer Value With 3D
This seminar explores the use of 3D graphics in UIs on mobile devices
• Inspirational examples• Framework for analyzing and working with 3D• What’s the real customer value of 3D?• Provide a 3D experience from low-end to high-end devices• A 3D roadmap to the future• TAT designs & concepts for you
Typically the palette of "3D techniques" has been comprised of lighting, shadows, focus, depth, camera angles and similar visual elements. At TAT, we believe that 3D is more than 3D graphics.
We understand "3D" in user interfaces as a paradigm where the appearance of the interface displays three-dimensional or real-world qualities, while the technology behind the visualization is not necessarily built up as a real 3 dimensional simulation. To benefit from the full potential of 3D in mobile user interfaces we also need to consider the total user experience.
Android Seminar & Workshop
TAT presents an inspiring look at the options with Android today and tomorrow.
• Inspirational examples for you• Differentiating your UI on Android• Configurations for 3D, hardware acceleration, silo apps, more• Suggested roll-out and strategy for this new platform• How does the Cascades UI Framework enhance Android?• How to get better performance and enhance hardware efficiency• Hands-on demonstration of Android development
The new kid on the block is making waves, but how do we harness all that Android has to offer? What is Android? What can it do for you today? What can it do for you tomorrow?
By providing the interaction design and the visual look & feel of the G1, TAT has been part of Android right from the beginning. Already we have deep experience and insight into both the technical and UX opportunities available and can help you push the limits of 3D, unique transitions, work processes, prototyping, and more.
Mixing pragmatism and inspiration TAT will help you address key issues for your organization with Android.
Multimodal UIs - User Experience and WOW
An exploration of a cohesive use of sound, haptics, & animation
• Inspirational examples• How can sound and haptics enhance UX?• A taxonomy for multimodal UIs• Usability factors & aesthetic concerns• Synchronizing animations with sound and haptics• TAT designs & concepts for you
Like films and computer games, the mobile UI can become more engaging and immersive by including sounds and haptics.
Sound and haptics can improve usability on touchscreen devices where visual feedback is not always sufficient.
To achieve a true WOW factor, sound, haptics, and visuals must all be synchronized together.
ProCams and the Future of Mobile Displays
Novel displays and the potentials of ProCams (projectors combined with cameras)
• Inspirational examples• Overview of emerging display technologies• Opportunities and challenges of new displays• Different aspects of interaction with projected UIs• New interaction paradigms of flexible and transparent displays• TAT designs & concepts for you
Projected graphics will enable mobiles to become far more social than today. Collaboration can be enabled by using the projection as input surface
Since pressing buttons will interfere with projections, other ways of interaction, such as camera captured gestures or speech input, become necessary
Flexible and transparent displays open up new interaction possibilities, such as “bend to zoom”
Realism and Physics
New levels of realism in UIs enabled by physics modeling
• Inspirational examples• How can physics and realism enhance UX?• To what parts of a UI can physics be applied?• How to design realistic output that corresponds to input gestures• Benefits and drawbacks of realism• Simulating physics on low end and high end devices• TAT designs & concepts for you
Mobiles with touchscreen or motion sensor input can become more intuitive and impressive if graphics behave realistcally, modeling physics.
UIs that model physics are responsive to subtle gestures, which makes them inviting and playful.
When used right, realistic UIs can both enhance usability and add a considerable WOW-factor.
Thinking Beyond Flash Prototyping
How to capture creativity and design strong products through efficient prototyping
• Inspirational workflow examples• What is a sketch? What is a prototype?• An overview of prototyping tools and methods• Communicating ideas through a multitude of sketches• Useful skills for efficient prototyping• TAT process and tools for prototyping
Introducing new prototyping methods can open up new opportunities byincreasing product differentiation
At TAT we believe that many important aspects of new products and services can be designed and evaluated before entering a resource-heavy prototyping phase.
By introducing practices for improved sketching at your company, a more creative and flexible prototyping culture will emerge.
TAT believes that artefacts radically improve communication and that all aspects of products, services, processes and business models can be visualized.
THANK YOU!