deformable and shape changing interfaces
TRANSCRIPT
Deformable and shape-changing user interfaces
Kasper Hornbæk, University of Copenhagen
apple.com
wallviz.dk
Samsung, CES 2013
PaperPhone, CHI 2012
Samsung, CES 2014
MorePhone
MorePhone, CHI 2013
PneUI, UIST 2013
PneUI, UIST 2013
Rasmussen et al., CHI 2012
44 papers on shape change
44 papers on shape change
Little UX
ghost-fet.org
ghost-fet.org
Dr. Kasper Hornbæk Dr. Esben W. Pedersen Giovanni M. Troiano John Tiab
UserExperience
Technology Design
GHOST
UserExperience
Technology Design
GHOST
Study 1:Guessability of Gestures
Study 2:Crowd-Sourced Reactions
Study 3:Repository Grid Evaluation
Troiano, Pedersen & Hornbæk (AVI 2014)
Gestures were recorded with four cameras
Gestures were recorded with four cameras
10 page coding manual
Many%varia)ons%
Influence of Elasticity
Study 1:Guessability of Gestures
Study 2:Crowd-Sourced Reactions
Study 3:Repository Grid Evaluation
Pedersen, Subramanian & Hornbæk (CHI 2014)
Pedersen, Subramanian & Hornbæk (CHI 2014)
Shape-changing Handheld
Shape-changing Handheld
Parameters of shape change
1.5x
3x
5x
pi
-pi/2
pi/2
15 mm
20 mm
60 mm
n = 3
n = 8
n = 14
n = 1
n = 2
n = 3
towards (18 mm)
away (18 mm)
13 mm/s
0 1 2 3 64 5
53 mm/s
0 1 2 3 64 5
Area Curvature Amplitude Zero-crossing Speed Tapering Corner bend
LowM
ediumH
igh
Hemmert et al. Gomes et al. Roudaut et al.
Parameters of shape change
1.5x
3x
5x
pi
-pi/2
pi/2
15 mm
20 mm
60 mm
n = 3
n = 8
n = 14
n = 1
n = 2
n = 3
towards (18 mm)
away (18 mm)
13 mm/s
0 1 2 3 64 5
53 mm/s
0 1 2 3 64 5
Area Curvature Amplitude Zero-crossing Speed Tapering Corner bend
LowM
ediumH
igh
Hemmert et al. Gomes et al. Roudaut et al.
Parameters of shape change
1.5x
3x
5x
pi
-pi/2
pi/2
15 mm
20 mm
60 mm
n = 3
n = 8
n = 14
n = 1
n = 2
n = 3
towards (18 mm)
away (18 mm)
13 mm/s
0 1 2 3 64 5
53 mm/s
0 1 2 3 64 5
Area Curvature Amplitude Zero-crossing Speed Tapering Corner bend
LowM
ediumH
igh
Hemmert et al. Gomes et al. Roudaut et al.
Parameters of shape change
1.5x
3x
5x
pi
-pi/2
pi/2
15 mm
20 mm
60 mm
n = 3
n = 8
n = 14
n = 1
n = 2
n = 3
towards (18 mm)
away (18 mm)
13 mm/s
0 1 2 3 64 5
53 mm/s
0 1 2 3 64 5
Area Curvature Amplitude Zero-crossing Speed Tapering Corner bend
LowM
ediumH
igh
Hemmert et al. Gomes et al. Roudaut et al.
Use scenariosNotification Approach
Use scenariosNotification Approach
187 participants
AttrakDiff2 Animacy SAM UC Specific
Zero crossing
Area
Curvature
Amplitude
Tapering
Corner bend
High
Low
Prag
mat
ic Q
ualit
y
Zero crossing
Area
Curvature
Amplitude
Tapering
Corner bend
High
Low
Prag
mat
ic Q
ualit
y
Structured
Simple
Practical
Predictable
PQ
Zero crossing
Area
Curvature
Amplitude
Tapering
Corner bend
High
Low
Prag
mat
ic Q
ualit
y
Zero crossing
Tapering
Area
Curvature
Amplitude
Corner bend
High
Low
Hed
onic
Qua
lity
Zero crossing
Tapering
Area
Curvature
Amplitude
Corner bend
High
Low
Hed
onic
Qua
lity
Captivating
Stylish
Premium
Creative
HQ
Corner Bend
27 %less urgent than the three most urgent shape changes
Corner Bend
27 %less urgent than the three most urgent shape changes
Corner Bend
27 %less urgent than the three most urgent shape changes
Amplitude Zero crossingArea
Calm
Aroused
PleasentUnp
leasan
t
III
III IV
Calm
Aroused
PleasentUnp
leasan
t
III
III IV
alarmedtense
angryafraid
annoyeddistressed
frustrated
miserable
sadgloomy
depressed
bored
droopytired sleepy
relaxedsatisfied
at easecontentserene
gladpleased
happy
delighted
excited
astonishedaroused
Calm
Aroused
PleasentUnp
leasan
t
III
III IV
alarmedtense
angryafraid
annoyeddistressed
frustrated
miserable
sadgloomy
depressed
bored
droopytired sleepy
relaxedsatisfied
at easecontentserene
gladpleased
happy
delighted
excited
astonishedaroused
Calm
Aroused
PleasentUnp
leasan
t
III
III IV
alarmedtense
angryafraid
annoyeddistressed
frustrated
miserable
sadgloomy
depressed
bored
droopytired sleepy
relaxedsatisfied
at easecontentserene
gladpleased
happy
delighted
excited
astonishedaroused
Calm
Aroused
PleasentUnp
leasan
t
III
III IV
alarmedtense
angryafraid
annoyeddistressed
frustrated
miserable
sadgloomy
depressed
bored
droopytired sleepy
relaxedsatisfied
at easecontentserene
gladpleased
happy
delighted
excited
astonishedaroused
Calm
Aroused
PleasentUnp
leasan
t
III
III IV
alarmedtense
angryafraid
annoyeddistressed
frustrated
miserable
sadgloomy
depressed
bored
droopytired sleepy
relaxedsatisfied
at easecontentserene
gladpleased
happy
delighted
excited
astonishedaroused
area
Area
Calm
Aroused
PleasentUnp
leasan
t
III
III IV
alarmedtense
angryafraid
annoyeddistressed
frustrated
miserable
sadgloomy
depressed
bored
droopytired sleepy
relaxedsatisfied
at easecontentserene
gladpleased
happy
delighted
excited
astonishedaroused
area
Amplitude
Calm
Aroused
PleasentUnp
leasan
t
III
III IV
alarmedtense
angryafraid
annoyeddistressed
frustrated
miserable
sadgloomy
depressed
bored
droopytired sleepy
relaxedsatisfied
at easecontentserene
gladpleased
happy
delighted
excited
astonishedaroused
area
amplitude
Amplitude
Calm
Aroused
PleasentUnp
leasan
t
III
III IV
alarmedtense
angryafraid
annoyeddistressed
frustrated
miserable
sadgloomy
depressed
bored
droopytired sleepy
relaxedsatisfied
at easecontentserene
gladpleased
happy
delighted
excited
astonishedaroused
area
amplitude
Corner bend
Calm
Aroused
PleasentUnp
leasan
t
III
III IV
alarmedtense
angryafraid
annoyeddistressed
frustrated
miserable
sadgloomy
depressed
bored
droopytired sleepy
relaxedsatisfied
at easecontentserene
gladpleased
happy
delighted
excited
astonishedaroused
area
amplitude
Corner bend
corner bend
Calm
Aroused
PleasentUnp
leasan
t
III
III IV
alarmedtense
angryafraid
annoyeddistressed
frustrated
miserable
sadgloomy
depressed
bored
droopytired sleepy
relaxedsatisfied
at easecontentserene
gladpleased
happy
delighted
excited
astonishedaroused
area
amplitude
corner bend
Calm
Aroused
PleasentUnp
leasan
t
III
III IV
alarmedtense
angryafraid
annoyeddistressed
frustrated
miserable
sadgloomy
depressed
bored
droopytired sleepy
relaxedsatisfied
at easecontentserene
gladpleased
happy
delighted
excited
astonishedaroused
area
amplitude
corner bend
taperingcurvature
zero crossing
Calm
Aroused
PleasentUnp
leasan
t
III
III IV
alarmedtense
angryafraid
annoyeddistressed
frustrated
miserable
sadgloomy
depressed
bored
droopytired sleepy
relaxedsatisfied
at easecontentserene
gladpleased
happy
delighted
excited
astonishedaroused
area
amplitude
corner bend
taperingcurvature
zero crossing
Calm
Aroused
PleasentUnp
leasan
t
III
III IV
alarmedtense
angryafraid
annoyeddistressed
frustrated
miserable
sadgloomy
depressed
bored
droopytired sleepy
relaxedsatisfied
at easecontentserene
gladpleased
happy
delighted
excited
astonishedaroused
area
amplitude
corner bend
taperingcurvature
zero crossing
Study 1:Guessability of Gestures
Study 2:Crowd-Sourced Reactions
Study 3:Repository Grid Evaluation
Kwak, Markopolos, Bruns & Hornbæk (DIS 2014)
Bootcamp I: Master Students Industrial Design
Goal:
Define interaction possibilities and pilot workshop approach
Results:
10 prototypes
- 3 -
isolated pairs of user input and system output. However, when the properties of a product, such as its shape, change our perceptions will follow these changes through a process of adaptation [12]. A product that changes its shape can potentially give rise to a feedback loop playing on the action-perception possibilities, offering a dynamic interaction between the action of the user and the reaction of the product, which has been named interactive materiality [34]. Our perspective on designing shape-changing interfaces and evaluating emerging interactions takes this broader perspective.
The focus is on interaction supported by manipulation, gesture, and touch, with medium sized (hand scale) objects. Lederman and Klatzky [28] provided a systematic framework for analyzing hand movements and manipulations that are conducted in the context of the active explorations people make in order to perceive with touch, what Gibson [12] called active touch. Specifically, Lederman and Klatzky [28] identified the necessity and demonstrated the appropriateness of a small set of stereotypical movements for perceiving the shape of different objects: lateral motion, pressure, static contact, unsupported holding, enclosure, contour following, function test, and part motion test. This framework is not directly related to shape-changing interfaces or current models thereof; yet it helps generate design ideas and identify ways one may engage with shape-changing interfaces.
Design Processes In a one week-long design boot camp, we invited 20 advanced Master level students in industrial design to use a parallel design process to develop shape-changing interfaces that feature variations on the interactive action-perception loop. The shape-changing interfaces were to sense the explorations that the user makes through active touch [12] and adapt their physical properties. Students
were instructed to design artifacts that support different explorations of users along the classification of typical movement patterns for exploratory procedures (EPs). A collection of ten shape-changing artifacts was created (Figure 1). Videos of the artifacts can be viewed at http://www.anonymous.com.
These first design explorations were followed by a more sustained and in-depth design process, where a design researcher synthesized the initial design concepts, redesigned the initial artifacts in order to create a collection of six shape changing interfaces featuring very distinct experiences of shape-change.
The processes described above resulted in six artifacts, illustrated in Figure 2 through Figure 7. The accompanying video details the interaction possible with the artifacts. These six artifacts were used in the study presented next.
Figure 1. First generation of shape-changing interfaces, exploring dynamic orientation, form, volume, texture, and spatiality in one and two degrees of freedom. The bottom-
right figure shows two artifacts.
Figure 2. PIEGA explores dynamic changes in form as a response to static contact and consists of a cloth surface
attached in the center to a servomotor. The rotation of the servomotor results in folds in the cloth surface. A capacitive sensor is used to determine user presence and static contact.
Figure 3. GATO explores dynamic changes in texture as a response to static contact and consists of a platform with spring steel loop ‘hairs’ that lowers and rises to alter the length of the ‘hairs’; short hairs are rougher to the touch than longer hairs.
A capacitive sensor is used to determine user presence and static contact.
COGNITIVE PSYCHOLOGY 19, 342-368 (1987)
Hand Movements: A Window into Haptic Object Recognition
SUSAN J. LEDERMAN
Queen’s University at Kingston, Ontario, Canada
AND
ROBERTA L. KLATZKY
University of California at Santa Barbara
Two experiments establish links between desired knowledge about objects and hand movements during haptic object exploration. Experiment 1 used a match- to-sample task, in which blindfolded subjects were directed to match objects on a particular dimension (e.g., texture). Hand movements during object exploration were reliably classified as “exploratory procedures,” each procedure defined by its invariant and typical properties. The movement profile, i.e., the distribu- tion of exploratory procedures, was directly related to the desired object knowl- edge that was required for the match. Experiment 2 addressed the reasons for the specific links between exploratory procedures and knowledge goals. Hand move- ments were constrained, and performance on various matching tasks was as- sessed. The procedures were considered in terms of their necessity, sufficiency, and optimahty of performance for each task. The results establish that in free exploration, a procedure is generally used to acquire information about an object property, not because it is merely sufficient, but because it is optimal or even necessary. Hand movements can serve as “windows,” through which it is pos- sible to learn about the underlying representation of objects in memory and the processes by which such representations are derived and utilized. o 1987 Academic
Press. Inc.
When we feel extremely helpless in a situation, we commonly say, “My hands are tied!” Indeed, it is hard to imagine a world in which we cannot feel the soft fur of a kitten or even tie our shoelaces. Yet, psy- chology has often portrayed the hand as a second-class citizen. Research
The research reported in this paper was supported by the Natural Sciences & Engineering Research Council of Canada (Grant A9854 awarded to SJL) and by the National Science Foundation (Grant BNS84-21340 awarded to RLK). Reprint requests may be sent to SJL, Psychology, Queen’s University, Kingston, Ontario, Canada K7L 3N6 or to RLK, Psy- chology, University of California, Santa Barbara, CA 93106. Order of authorship does not reflect relative contribution; both authors contributed equally to the work. We thank An- drew Currie for his considerable contribution; he helped to prepare the stimulus objects, ran the experiments, scored some of the videotapes for our reliability checks, collated and analyzed much of the data, and provided valuable comments in general discussion.
342 OOlO-0285/87 $7.50 Copyright 0 1987 by Academic Press, Inc. All rights of reproduction in any form reserved.
Design Interaction Behaviour
Repertory Grid Study
These two seem insecure
This one seemsdetermined
Insecure Determined
Constructs
ConstructsFriendly/Distant
Indifferent/Attentive
Honest/Underhand
Random/Predictable
Sturdy/FragileMechanical/Organic
Cautious/Ambitious
Insecure/Determined
Unconcerned/InsecureVulnerable/Strong
Surly/Sensitive
Rebellious/Cooperative
Stubborn/Compliant
Obedient/Recalcitrant
Open/MysteriousBoisterous/Introvert
Social/Introvert
Skittish/Affectionate
Frightened/Defensive
Aggressive/Coy
Weak/Powerful
Playful/Calm
Restful/Hasty
Downhearted/HopefulHappy/Sad
Future Work
- How do we map deformation to (sound) parameters?- Participatory Design with 10 musicians as co-designers.
Deformable UI for real people
Workshops continued
Deformable and shape-changing user interfaces
Kasper Hornbæk, University of Copenhagen